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The  development  of  an  improved  means  for  analyzing  multi-task  and  multi-criterion  data  has 
been  a  concern  for  at  least  three  decades.  The  Rotated  Diagonal  Factors  (RDF)  approach  presented 
here  is  an  outgrowth  of  the  general  dissatisfaction  with  the  Multivariate  Analysis  of  Variance 
(MANOVA)  approach.  The  MANOVA  approach  is  that  it  fails  to  address  important  and  critical 
issues  regarding  correlations  among  tasks  and  criteria.  Some  sort  of  factor  analytic  approach  is 
desirable  to  identify  underlying  dimensions  that  would  be  useful  in  understanding  those 
relationships.  Theoretically,  there  should  be  at  least  three  types  of  independent  factors  that  might  be 
found.  The  first  would  be  a  between-tasks  type  that  could  account  for  significant  criterion 
correlations  between  different  tasks.  The  second  would  be  a  within-task  type  that  could  account  for 
significant  remaining  relationships  among  different  criteria  for  a  single  task  not  already  accounted 
for  by  the  first  type  of  factor.  The  final  would  be  a  within-criterion  type  that  would  account  for  the 
remaining  variance  of  a  criterion  not  accounted  for  by  the  first  two  types.  After  identifying  these 
types  of  independent  factors,  it  would  be  desirable  to  establish  if  individual  differences  and 
experimental  manipulations  had  significantly  impacted  all  three  types  of  factors. 

I  wish  to  thank  Dr.  Floyd  Glenn,  CHI  Systems,  Inc.,  for  his  encouragement.  This  work  was 
funded  by  the  Naval  Air  Warfare  Center  Aircraft  Division,  Patuxent  River,  Maryland. 

A  method  for  mathematically  rotating  factors  to  the  desired  structure  is  being  developed.  This 
improvement  will  eliminate  the  necessity  for  manual  graphical  factor  rotation,  which,  in  turn,  will 
make  the  RDF  approach  completely  objective.  Another  enhancement  which  is  under  development  is 
a  knowledge-based  program  for  the  automatic  interpretation  of  RDFs.  This  enhancement  would 
permit  the  automatic  recognition  of  RDFs  that  represent  such  concepts  as  "time-sharing,"  "attention 
shifting  among  tasks,"  "response  speed,"  "response  accuracy,"  "speed-accuracy  trade-offs,"  etc. 
While  these  developments  have  not,  as  yet,  been  completed,  the  basic  concepts  and  procedures  for 
the  RDF  approach  have  now  been  developed  and  tested  to  a  point  where  they  can  be  used  by  other 
researchers  who  are  faced  with  the  analysis  of  multi-task  and  multi-criterion  data. 
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1.  INTRODUCTION 

Many  human  factors  studies  involve  performance  of  simulated  tasks  under  varying 
experimental  conditions.  Major  questions  raised  in  these  studies  almost  always  concern  how 
much  the  experimental  conditions  enhance  or  degrade  task  performance  and  whether 
performance  differences  are  statistically  significant.  Analysis  of  Variance  (ANOVA),  the 
method  traditionally  used  to  determine  this  impact,  partitions  total  task  performance  into 
independent  components.  Historically,  the  major  variance  components  have  included  subject 
effects,  (i.e.,  variance  attributable  to  individual  or  group  differences),  experimental  effects 
(i.e.,  variance  attributable  to  various  experimental  manipulations),  and  residual  effects 
(i.e.,  unexplained  variance  usually  attributed  to  error).  The  ANOVA  technique  also  computes 
model  coefficients  for  various  levels  of  subjects  and  experimental  manipulations  and  tests  their 
significance  to  determine  the  probability  those  differences  happened  by  chance  alone. 

A  new  analytic  technique  to  be  discussed  and  demonstrated  in  this  paper  is  one  that 
extends  the  concept  of  partitioning  variance  into  yet  further  independent  components  when  the 
study  involves  multiple  tasks  and  multiple  performance  criteria.  It  combines  traditional 
multivariate  ANOVA  techniques  and  factor  analytic  techniques.  The  technique  is  referred  to  as 
the  Rotated  Diagonal  Factors  (RDF)  approach.  It  is  expected  to  be  particularly  useful  for 
situations  when  performance  on  a  single  task  is  measured  on  more  than  one  criterion,  when 
performance  on  more  than  one  task  is  measured,  or  a  combination  of  these  situations. 

1.1  Need  to  Consider  Multiple  Task  Situations 

Experimenters  often  assume  that  effects  in  laboratory  studies  of  a  single  task  will  also  be 
obtained  when  that  task  is  accomplished  in  the  real  world.  More  often  than  not,  however,  an 
isolated  task  studied  under  laboratory  conditions  is  only  one  of  several  ongoing  tasks  for  which 
an  operator  may  be  responsible  under  real  world  situations.  While  task  demands  influence  what 
needs  to  be  done,  operators  ultimately  determine  how  much  attention  they  dedicate  to  each 
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ongoing  task  responsibility.  Indeed,  ability  to  rapidly  adapt  behavior  to  momentary  task-related 
attentional  demands  is  one  of  the  strengths  of  human  operators.  This  ability  to  time-share 
attention  among  several  tasks  suggests  that  conclusions  based  on  situations  in  which  only  a 
single  task  was  performed  may  not  be  applicable  to  real  world,  multi-task  conditions.  If  true, 
task  performance  should  be  studied  in  the  context  of  the  totality  of  tasks  to  be  performed.  It 
seems  obvious  that  specific  experimental  manipulations  (e.g.,  different  display/control  designs, 
crewstation  layouts,  seating,  restraints,  etc.)  may  enhance  performance  on  one  task,  but,  at  the 
same  time,  lead  to  degraded  performance  on  another.  Other  situational  conditions  (e.g.,  levels  of 
noise,  illumination,  vibration,  G-forces,  etc.)  may  differentially  enhance  or  degrade  performance 
across  several  tasks.  The  likelihood  of  such  outcomes  strongly  suggests  that  instead  of  merely 
considering  the  variance  in  performance  of  each  separate  task  in  isolation,  investigators  should 
analyze  their  data  for  potential  between-tasks  effects.  Between-tasks  effects  can  be  defined  as 
"those  effects  which  cause  performance  on  one  or  more  criteria  of  one  task  to  covary  (positively 
or  negatively)  with  performance  on  one  or  more  criteria  for  other  tasks." 

1.2  Need  to  Consider  Multiple  Criteria  for  a  Single  Task 

When  considering  performance  on  a  single  task,  conclusions  drawn  about  either 
individual  differences  or  experimental  manipulations  may  be  contingent  on  how  task 
performance  is  measured.  In  a  target  recognition  task,  for  instance,  analyzing  only  "percent  of 
correct  recognitions"  may  lead  to  quite  different  conclusions  than  the  analysis  of  "decision 
response  time."  Indeed,  even  "response  times  for  correct  decisions"  are  often  different  from 
"response  times  for  incorrect  decisions."  Additionally,  it  has  long  been  accepted  that  humans  are 
capable  of  time-accuracy  trade-offs,  and,  unless  the  experimenter  includes  multiple  criteria  for 
evaluating  task  performance,  the  impact  of  both  individual  differences  and  experimental 
manipulations  during  a  laboratory  study  may  not  be  appreciated  fully.  Thus,  even  when  studying 
only  a  single  task,  investigators  may  need  to  analyze  performance  variability  among  several 
criteria  for  that  task  to  determine  the  presence  of  significant  within-task  effects.  Within-task 
effects  can  be  defined  as  "those  effects  which  cause  performance  on  one  criterion  of  a  task  to 
CO  vary  (positively  or  negatively)  with  another  criterion  measure  of  that  same  task." 
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Finally,  in  addition  to  potential  between-tasks  effects  and  within-tasks  effects,  it  is  also 
possible  to  assume  that  individual  differences  or  some  experimental  manipulation  may  effect 
only  a  single  criterion  of  a  single  task.  This  type  of  effect  is  defined  here  as  a  within-criterion 
effect. 


Thus,  theoretically,  in  a  multi-task,  multi-criterion  study,  the  total  variance  in 
performance  on  each  criterion  should  be  able  to  be  divided  into  three  types  of  independent 
variance  components:  (a)  between-tasks,  (b)  within-task  (but  between  criteria),  and  (c)  within- 
criterion.  The  Rotated  Diagonal  Factors  approach,  to  be  discussed  later,  accomplishes  this 
objective. 

1.3  Individual  Differences  in  Task  Performance 

A  "proficient"  task  performer  is  usually  defined  as  one  who  can  respond  fairly  rapidly  and 
makes  few  or  no  errors.  Conversely,  one  who  responds  slowly  and  makes  many  errors  is  almost 
always  considered  to  be  a  "non-proficient"  task  performer.  If  subjects  (Ss)  in  a  study  were  only 
composed  of  these  two  styles  of  people,  then  performance  measures  of  response  time  and 
response  accuracy  would  always  be  negatively  related  across  subjects  (i.e.,  those  with  short 
response  times  would  have  high  accuracies  and  those  with  long  response  times  would  have  low 
accuracies).  However,  there  may  also  be  those  who  might  be  described  as  "careful"  individuals 
who  tend  to  make  few  errors  but  have  relatively  long  response  times  or  those  who  might  be 
described  as  "decisive"  and  who  respond  extremely  fast  but  make  somewhat  more  errors.  If  Ss  in 
a  study  were  only  composed  of  these  two  styles  of  people,  then  performance  measures  of 
response  time  and  response  accuracy  would  always  be  positively  related  across  subjects 
(i.e.,  those  with  high  response  times  would  have  high  accuracies  and  those  with  low  response 
times  would  have  low  accuracies).  Of  course,  in  most  studies,  all  four  styles  of  people  may  be 
present  and  the  direction  of  the  relationship  between  response  time  and  response  accuracy  is 
difficult  to  predict. 
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Task  difficulty  can  also  affect  the  relationship  between  response  time  and  response 
accuracy.  For  example,  more  difficult  decisions  may  take  more  time  and  have  a  higher 
likelihood  of  error.  The  greater  the  range  of  item  difficulty  on  a  particular  task,  the  more  likely  it 
is  for  response  time  and  response  accuracy  to  have  a  negative  relationship.  Also,  in  a  multi-task 
situation,  difficulty  level  for  one  task  may  influence  the  available  time  Ss  have  to  attend  to 
another  task. 

Amount  of  training  can  also  effect  the  relationship  between  response  time  and  response 
accuracy.  With  more  and  more  training  and  experience,  people  tend  to  become  more  proficient, 
but  individual  differences  between  "fully  trained"  persons,  caused  by  inherent  capabilities 
between  those  persons,  may  still  be  present.  Even  in  high  demand,  multi-task  situations,  humans 
can  learn  to  reeognize  the  urgency  and  criticality  of  decision  tasks  and  appropriately  trade-off 
response  time  for  response-accuracy.  They  will  usually  take  somewhat  longer  times  to  make 
more  difficult  and  critical  decisions,  but  they  can  also  usually  recognize  when  decisions  must  be 
made  even  though  they  might  have  preferred  more  time  to  consider  their  responses.  Thus,  the 
level  of  task  demands  may  well  affect  the  direction  and  magnitude  of  the  relationship  between 
response  time  and  response  accuracy. 

Any  specific  task  on  which  performance  can  be  measured  undoubtedly  requires  several 
different  types  of  processes  (e.g.,  perception  of  the  stimuli,  cognitive  processing  of  those  stimuli, 
production  of  a  response,  etc.).  When  two  different  tasks  require  similar  processes,  one  would 
expect  those  who  are  more  gifted  in  those  specific  capabilities  to  do  somewhat  better  on  both 
tasks  than  those  who  are  less  gifted.  Thus,  individual  differences  in  similarly  required  task 
processes  should  lead  to  positive  covariance  between  criteria  for  two  tasks.  Enhanced 
proficiency  in  performing  any  specific  task  could  also  arise  through  training  and  practice  on  that 
task.  Thus,  even  if  two  tasks  did  not  share  any  of  the  same  required  capabilities,  more  training 
and  practice  on  both  tasks  for  some  individuals  than  others  could  lead  to  positive  criterion 
covariance  between  them.  Finally,  it  is  reasonable  to  assume  that  the  level  of  performance 
produced  by  an  individual  on  any  task  could  also  be  effected  by  that  individual's  general  level  of 
motivation,  fatigue,  or  environmental  factors  such  as  levels  of  distraction.  Thus,  certain  general 
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effects,  not  directly  related  to  any  of  the  tasks,  could  also  result  in  positive  covariance  among 
criteria  for  two  tasks. 

1.4  The  Challenge  of  Multiple  Task  and  Multiple  Criterion  Studies 

From  the  foregoing  discussions,  it  follows  that  between-tasks  effects  may  be  found  which 
are  attributable  to  individual  differences  (either  those  that  result  from:  (a)  inherent  capabilities, 
(b)  differential  training  and  practice,  or  (c)  general  factors  such  as  motivation,  fatigue,  or 
distractions).  Each  of  these  could  cause  differences  in  the  level  of  task  proficiency  of  subjects 
which  result  in  positive  covariance  among  criteria  for  different  tasks.  Thus,  while  many 
between-tasks  factors  may  be  primarily  caused  by  individual  differences,  during  a  repeated 
measures  study,  specific  experimental  manipulations  (e.g.,  number  of  sessions,  length  of 
sessions,  order  of  conditions  within  sessions,  and  periods  within  sessions)  could  all  be  expected 
to  impact  the  extent  of  practice  and  conditions  of  fatigue  and,  therefore,  the  subject's  task 
proficiencies. 

Within-task  effects  could  also  result  from  individual  differences  in  capabilities  not  used 
in  other  tasks  and  could  also  represent  individual  differences  in  trading-off  speed  and  accuracy. 
A  particular  experimental  condition  might,  conceivably,  affect  how  one  particular  task  gets 
accomplished,  but  have  little  effect  on  other  tasks. 

If  human  factors  related  findings  are  expected  to  generalize  to  the  real  world,  then 
researchers  must,  for  many  situations,  investigate  performance  on  multiple  ongoing  tasks.  If  they 
are  to  fully  understand  how  behavior  is  being  affected  by  both  individual  differences  and  various 
experimental  manipulations,  then  they  must  also  include  multiple  criteria  for  all  of  those  tasks. 
While  MANOVA  offers  a  method  for  analyzing  multi-task  and  multi-criterion  effects,  it  is  not 
sufficient  to  isolate  and  identity  the  between-tasks,  within-task,  and  within-criterion  effects 
discussed  earlier.  The  RDF  approach  was  developed  for  this  specific  purpose. 


5 


NAWCADWAR-96-36-TR 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


6 


NAWCADWAR-96-36-TR 


2.  MULTIVARIATE  TECHNIQUES 

In  this  section,  similarities  and  differences  among  six  multivariate  techniques  used  in 
experimental  data  analysis  will  be  discussed.  These  multivariate  techniques  and  the  subsections 
in  which  they  will  be  discussed  are: 

2.1  Analysis  of  Variance  (ANOVA); 

2.2  Multiple  Correlation  (MC)  and  Multiple  Regression  (MR); 

2.3  Diagonal  Factor  Analysis  (DFA); 

2.4  Multivariate  Analysis  of  Variance  (MANOVA); 

2.5  Factor  Analysis  (FA)  and  Factor  Rotation  (FR)  methods;  and 

2.6  Canonical  Analysis  (CA) 

The  reason  for  describing  these  techniques  in  some  detail  is  that  the  RDF  approach,  which  will 
be  discussed  in  subsection  2.7,  is  an  extension  of  these  techniques  and  of  the  logic  and  principles 
upon  which  they  are  based. 

2.1  Analysis  of  Variance  (ANOVA) 

Textbooks  on  ANOVA  spend  much  time  showing  experimental  designs  which  cause  the 
main  effects  and  interaction  terms  to  be  independent  of  one  another,  how  to  determine  the 
degrees  of  freedom  associated  with  main  effects  and  interaction  terms,  how  to  calculate  the 
various  independent  portions  of  variance  associated  with  those  factors,  and  how  to  test  their 
significance.  The  lack  of  space  here  prohibits  any  detailed  discussion  of  the  ANOVA  approach, 
but  the  major  principles  underlying  ANOVA  will  be  discussed.  For  that  discussion,  an  ANOVA 
study  having  two  main  effects  (A  and  B)  and  an  interaction  term  (AB)  will  be  used  as  an 
example.  We  can  think  of  a  data  matrix  in  which  we  have  rows  representing  the  different  levels 
of  A,  columns  representing  the  different  levels  of  B,  and  the  cells  of  the  matrix  representing  the 
levels  of  AB  interactions  between  the  two  main  effects. 
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One  ANOVA  principle  is  that  the  two  main  effects  can  be  forced  to  be  independent  of 
one  another  by  requiring,  for  each  level  of  B,  an  equal  (or,  at  least,  proportional)  number  of  data 
cases  to  be  collected  for  a  given  level  of  main  effect  A. 


A  second  principle  used  by  ANOVA  is  that  the  actual  (obtained)  score  for  any  data  case 
is  simply  the  sum  of  an  overall  effect  (|i,  i.e.,  the  Greek  letter  "mu")  plus  the  sum  of  the  actual 
effects  of  each  specific  row,  column,  and  cell  and  an  error  term  for  that  data  case.  This  concept 
is  usually  expressed  as  the  "structural  equation"  for  a  particular  design.  For  the  case  of  two  main 
effects,  the  structural  equation  for  the  obtained  score  of  case  k  in  row  i,  column  j  would  be: 


Yij,  =  R  +  tti  +  P,  +ap,.  +  e,,,. 

This  equation  is  related  to  a  general  statistical  principle  that  the  mean  or  average  of  any  set  of 
numbers  is  the  best  prediction  (to  minimize  the  sum  of  the  errors  squared)  one  can  make  for  that 
set  of  numbers.  ANOVA  uses  this  principle  by  assuming  that  the  true  value  of  the  overall  effect 
or  any  level  of  main  effect  or  interaction  term  is  the  weighted  mean  of  the  matrix,  or  appropriate 
row,  column,  or  cell  of  the  matrix  (where  the  respective  means  are  weighted  by  the  number  of 
cases  in  the  respective  matrix,  row,  column,  or  cell).  If  one  calculates  the  overall  weighted  mean 
(p.)  of  all  cases  in  the  data  matrix,  it  would  be  the  best  prediction  one  could  make  if  all  one  knew 
was  that  the  data  case  came  from  that  sample.  In  ANOVA,  p  is  assumed  to  represent  the  true 
overall  effect.  Its  value  could  then  be  subtracted  from  each  data  case  score.  The  weighted 
residual  score  means  of  rows  and  columns  are  assumed  by  ANOVA  to  represent  the  true  effects 
for  the  various  levels  of  the  two  main  effects  (a;  and  Pj).  These  score  means,  for  each  row  and 
column,  are  then  subtracted  from  each  data  case  residual  score  in  that  row  and  column  and  the 
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means  of  each  matrix  cell  are  then  calculated.  These  residual  cell  means  are  assumed  by 
ANOVA  to  represent  the  true  effects  of  the  interaction  (apij)  between  the  various  levels  of  the 
two  main  effects.  These  terms  are  shown  below. 


main  effects 

R 

P. 

B  levels 

P.  ... 

Pj 

...  Pb 

a. 

aPii 

aPi2  ... 

aPu 

...  aP.B 

a, 

aPa. 

ap,,  ... 
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...  aP,B 

A 

... 

... 

... 

... 

...  ... 

levels 

tti 

apii 

otpb  ... 

aPij 

...  api3 

apAi 

apA2  ... 

«Pxi 

...  apAB 

Finally,  the  appropriate  cell  means  are  subtracted  from  each  residual  data  case  score  and 
the  result  is  assumed  by  ANOVA  to  be  the  actual  error  value  (e.  .^)  for  each  data  case. 

A  third  principle  used  by  ANOVA  is  a  general  statistical  proof  that,  in  the  long  run 
(i.e.,  if  an  infinite  number  of  random  samples  were  drawn),  the  true  variance  of  the  means  (o^m) 
of  random  samples  of  N  cases  all  drawn  from  the  same  population  will  be  the  true  variance  of 
that  population  (o^)  divided  by  the  number  of  cases  (N)  in  each  sample  (i.e.,  o^m  =  /  N  ). 

A  fourth  principle  used  by  ANOVA  is  also  a  general  statistical  proof  that,  if  an  infinite 
number  of  samples  of  N  cases  are  drawn  at  random,  the  expected  value  of  the  variance  (s^),  if 
multiplied  by  the  quantity  N/(N-1),  will  be  equal  to  the  true  population  variance  (cr^).  That  is. 

Average  (s"  "  N/(N-1)  )  =  a\ 

ANOVA  combines  these  two  principles  by  initially  assuming  that  the  levels  of  each  main 
effect  and  interaction  have  no  real  effect  (this  is  generally  referred  to  as  the  null  hypothesis).  The 
earlier  discussed  procedures  for  finding  the  various  means  permits  one  to  analyze  the  total 
variance  (hence  the  name  "Analysis  of  Variance")  into  independent,  additive  components 
(i.e.,  variance  attributable  to  the  first  main  effect  means,  variance  attributable  to  the  second  main 
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effect  means,  variance  attributable  to  the  means  of  the  interaction  of  the  main  effect,  and  variance 
attributable  to  "residual"  error).  If  the  null  hypothesis  is  true,  then  the  variance  of  each  set  of 
means  can  be  used  to  get  an  unbiased  estimate  of  the  true  population  variance.  That  is, 

2  „  2  ,  2  ,  2,2 
S  Y  -  S  „  +  S  p  +  S  +  S  error  • 

The  traditional  ANOVA  equations  for  computing  what  are  referred  to  as  the  sums  of 
squares  are  those  that  compute  functions  of  the  variance  of  the  appropriate  sets  of  means  (i.e..  A, 
B,  AB,  and  error).  The  equations  for  computing  what  are  referred  to  as  mean  squares  are  those 
which  derive  estimates  of  the  population  variance  based  on  the  number  of  the  means  being 
considered.  The  F-ratios  computed  by  ANOVA  are  simply  the  ratios  of  appropriate  mean 
squares  (i.e.,  population  variance  estimates)  and  are  referred  to  as  such  because  the  F  statistic 
reports  the  likelihood  of  such  ratios  when  the  null  hypothesis  is  true.  That  is,  F  tables  show  the 
probability  of  the  ratio  of  two  independent  estimates  of  variances  based  on  known  number  of 
cases  drawn  from  the  same  normally  distributed  population,  being  as  large  as  the  tabled  values. 

2.2  Multiple  Correlation  (MC)  and  Multiple  Regression  (MR) 

Let  us  suppose  that  the  correlation  between  two  variables  (Y  and  X)  is  calculated  by  a 
variation  of  the  Pearson  Product-Moment  Correlation  equation  (i.e.,  ryx  =  zyZx/N)  where  zy  and 
zx  are  the  standardized  scores  for  variables  Y  and  X  respectively,  and  N  is  the  number  of  cases  in 
the  sample.  It  can  be  shown  that,  when  this  is  done,  the  square  of  that  correlation  coefficient 
(i.e.,  r^yx)  yields  the  proportion  of  variance  of  variable  Y  (a  criterion  variable)  that  can  be 
explained  by  variable  X  (a  predictor  variable).  It  can  also  be  shown  that  the  "best"  linear 
prediction  (where  "best"  is  defined  as  that  which  minimizes  the  sum  (or  average)  of  the  squared 
errors  of  the  predictions)  of  zy  (i.e.,  z'  y)  will  be  ryx  zx. 
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Multiple  correlation  is  simply  an  extension  of  simple  correlation  for  cases  in  whieh  scores 
for  a  criterion  variable,  Y,  are  predicted  using  more  than  one  predictor  variable  (e.g.,  X],  X2,  X3, 
etc.).  That  is,  letting  P  represent  a  weight  for  multiplying  standard  scores,  then  the  equation  for 
Y's  predicted  standard  score  (z'y)  for  case  n  will  be: 


Z'vn  “  pi  Zxin  +  P2  ZX2n  +  —  +  Pm  Zxmn  • 


Thus,  the  purpose  of  multiple  correlation  is  to  derive  weights  for  multiplying  the  standard  scores 
of  the  predictor  variables  so  as  to  obtain  a  prediction  of  the  standard  scores  of  the  criterion 
variable,  Y,  which  minimizes  the  sum  (or  average)  of  the  squared  errors  of  prediction.  Because 
the  means  and  standard  deviations  of  the  criterion  and  all  of  the  predictor  variables  and  the 
criterion  are  known,  it  is  relatively  simple,  once  the  "best"  standard  score  weights  are  derived,  to 
then  calculate  a  raw  score  prediction  equation  for  predicting  the  raw  score  of  variable  Y  for  case 
n.  That  is,  letting  Y'  be  the  predicted  raw-score  of  variable  Y  and  substituting  (Xjn-  Mxi)  /  sxi  for 
each  zxin  and  (Y'n  -  My)  /  sy  for  z'yn  in  the  above  equation: 

(Y’n-  My)  /  Sy  =  Pi(X,n-Mxi)/Sxi+p2(X2„-Mx2)/Sx2+  ...  +Pm(X„n-Mxm)/SXm  . 


Multipl5dng  both  sides  by  sy  and  adding  My  to  both  sides. 


Y'n  -  My+PiSy/Sxi(Xin-Mxi)+P2Sy/Sx2(X2n-Mxi)+...+PmSy/Sxm(Xmn-Mxi)  . 


Letting  B.  =  p.sy/sxi,  collecting  constants  together. 


Y'n  =  My  +  B,(X,n-X,)  +  B2  (X2n-X2)  +  ...  +  Bn,  (X„,n-Xn,)  , 

=  My  -  (BiMxi+B2Mx2+...+BmMxm)  +  BiXin+B2X2n+  ...+  BmXmn  . 
And  letting  A  =  My  -  (BiMxi+  B2MX2  +  ...  +  BmMxm)?  then 

Y'n  =  A  +  (Bi  Xin  +  B2  X2n  +  ...  +  Bm  Xmn  )  . 


11 


NAWCADWAR-96-36-TR 


The  only  difference  between  multiple  correlation  (MC)  and  multiple  regression  (MR)  is 
that,  in  doing  MC,  raw  data  for  both  the  predictors  and  the  criterion  variable  are  treated  as  if  they 
had  first  been  converted  to  standard  scores.  Because  both  techniques  are  mathematically 
identical,  the  derived  raw  score  or  standardized  weights  and  results  of  testing  their  significance 
will  be  identical  regardless  of  whether  one  uses  MC  or  MR. 

The  correlation  between  the  actual  and  predicted  criterion  scores  (i.e.,  ryv)  would  yield 
what  is  referred  to  as  the  multiple  correlation  coefficient.  To  indicate  that  Y'  is  composed  of 
weighted  portions  of  m  predictor  variables,  rather  than  using  ryv,  the  multiple  correlation 
coefficient  is  symbolized  as  Ry  xi,x2,---,xm  ,  or  Ry  1,2,. ..,m  ,  or  more  simply,  Ry  1  .m,  or  even 
more  simply  as  R.  Its  square,  R^,  is  equal  to  the  amount  of  Y  variance  that  can  be  accounted  for 
by  the  prediction  equation.  The  final  multiple  correlation  coefficient  can  be  tested  for 
significance  using  an  F-test  where: 


F(df=„,,N-m-i)  =  (R^  /  m )  /  ( ( 1  -  r2)  /  (N-m-1) ) . 


It  can  be  seen  that  the  F  value  is  the  ratio  of  the  total  explained  variance  (i.e.,  R  )  divided  by  the 
number  of  predictor  variables  (i.e.,  m)  and  the  unexplained  variance  (i.e.,  1-R  )  divided  by  N-1 
minus  the  number  of  predictor  variables.  But  each  new  predictor  variable  selected  represents  a 
potentially  new  independent  source  of  explained  variance  and  can  be  tested  separately.  The 
significance  of  the  kth  predictor  is  also  determined  by  an  F-test.  The  numerator  of  this  test  is  the 
difference  in  variance  explained  by  the  total  k  variables  and  that  which  had  previously  been 
explained  by  k-1  variables.  The  denominator  for  this  test  is  the  unexplained  variance  based  on 
all  k  variables  divided  by  its  appropriate  degrees  of  freedom.  That  is, 

F(df:  l,N-k-l)  =  ((R\  -  R\  l,....k-l)  /  1)  /  ((  1  -  l.....k)/(N-k-l))  , 

(R^y  -  R\  /  (( 1  -  R^  ,,.,..k)/(N-k-l)) . 
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A  basic  problem  recognized  in  both  MC  or  MR  is  determining  how  many  predictor 
variables  to  include  in  the  prediction  equation.  There  are  three  different  general  solutions  to  this 
problem.  The  first  solution's  approach  is  referred  to  as  the  method  of  accretion  (or  "test 
selection").  It  first  selects  the  best  single  predictor  (i.e.,  the  one  that  will  account  for  the  most 
criterion  variance),  removes  the  effect  of  that  variable  from  the  matrix,  and  then  looks  for  the 
next  predictor  that  will  explain  the  most  residual  varianee  of  the  criterion,  and  so  on.  Each 
succeeding  F  value  is  computed  and  can  be  tested  by  the  above  equation.  If,  at  any  time,  in  this 
sequential  approach  the  amount  of  newly  explained  variance  is  not  deemed  to  be  significant,  then 
the  method  stops  at  that  point  without  selecting  the  non-significant  variable. 

The  second  solution's  approach  is  referred  to  as  the  method  of  deletion.  It  selects  all 
predictors  and  then,  again  using  a  variation  of  the  above  equation,  tests  the  significance  of  each 
predictor  variable.  If  one  or  more  predictor  variables  is  non-significant,  the  least  significant 
variable  is  eliminated  and  the  entire  process  is  then  repeated  without  that  variable.  This  process 
continues  until  all  predictor  variables  being  used  show  a  significant  contribution.  It  is  of  passing 
interest  that  the  first  two  methods  usually  arrive  at  the  same  conclusions  as  to  which  variables 
should  be  selected  for  the  prediction  equation. 

The  third  solution's  approach  can  be  referred  to  as  the  select  all  predictors  method.  It 
selects  all  predictors,  regardless  of  whether  they  are  significant  or  not,  and  reports  their  levels  of 
significance.  This  third  method  corresponds  closely  to  traditional  ANOVA  in  that  ANOVA  also 
uses  all  possible  predictors  (i.e.,  all  levels  of  main  effects  and  interaction  terms  in  its  structural 
equation).  One  impact  of  this  is  that  the  structural  equation  values  derived  by  ANOVA  may  be 
over-fitting  error  and,  thus,  may  not  be  as  effective  in  predicting  the  criterion  in  future  samples  of 
similar  data. 

Many  people  think  that  MC  and  ANOVA  are  different  ways  to  analyze  data.  Such  is  not 
the  case.  While  ANOVA  equations  for  obtaining  sums  of  squares  and  mean  squares  are 
interesting  from  an  historical  standpoint,  MC  can  be  used  to  accomplish  the  same  purpose  as  the 
traditional  ANOVA  approach.  To  accomplish  ANOVA  using  MC  requires  the  creation  of 
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dichotomous  variables  for  each  degree  of  freedom  in  the  ANOVA  model.  Data  for  any 
dichotomous  variable  is  coded  as  "0"  if  the  trait  in  question  is  not  present  and  "1"  if  the  trait  is 
present.  For  example,  if  three  levels  of  factor  A  are  used  in  a  given  study,  any  two  of  the  three 
levels  could  be  used  as  the  basis  for  eoding  the  needed  two  dichotomous  variables.  For  example, 
one  variable  to  represent  "Level  Ai"  and  one  variable  to  represent  "Level  A2"  could  be  created. 
Suppose,  in  this  same  study,  there  is  a  second  factor,  B,  which  also  has  three  levels.  Two 
dichotomous  variables  (Bi  and  B2)  to  represent  this  factor  could  also  be  created.  Finally,  four 
dichotomous  variables  to  represent  the  interaction  of  these  variables  (i.e.,  AiB|,  A1B2,  A2B1,  and 
A2B2)  could  be  created.  The  coded  data  for  the  eight  created  dichotomous  variables  for  all 
possible  combinations  of  the  different  levels  of  condition  and  order  are  shown  below.  It  can  be 
seen  that  each  combination  gets  assigned  a  different  set  of  "0"  and  "1"  values  across  the  eight 
dichotomous  variables,  even  though  specific  dichotomous  variables  do  not  exist  for  A3,  B3,  A1B3, 
A2B3,  or  A3B1,  A3B2,  or  A3B3. 

created  dichotomous  (predictor)  variables 
A  Level  B  Level  Ai  A2  B]  B2  AiBj  A1B2  A2B1  A2B2 
1  1  10  10  10  0  0 
1  2  10  0  10  10  0 

1  3  10000000 

2  1  0  1  1  0  0  0  1  0 

2  2  0  1  0  1  0  0  0  1 

2  3  01000000 

3  1  00100000 

3  2  00010000 

3  3  00000000 

It  is  worth  noting  that  no  information  is  lost  by  not  having  dichotomous  variables  for 
"Level  A3,"  "Level  B3,"  or  any  of  the  interactions  of  those  levels.  The  same  rationale  which 
permits  elimination  of  those  levels  is  identical  to  the  rationale  in  ANOVA  for  "Effect  A"  having 
only  two  degrees  of  freedom  (i.e.,  dfA  =  (a-1)),  "Effect  B"  having  only  two  degrees  of  freedom 
(i.e.,  dfe  =  (b-1)),  and  the  interaction  of  those  terms  having  only  four  degrees  of  freedom 
(i.e.,  dfAB  =  (a-l)(b-l)). 


14 


NAWCADWAR-96-36-TR 


To  accomplish  MC,  correlation  coefficients  among  these  predictor  variables  (Xs)  and  the 
criterion  variable  (Y)  are  first  computed.  Multiple  correlation  is  then  accomplished  using  the 
predictors  to  explain  the  criterion  variable.  In  performing  the  MC  approach,  the  R  will  keep 
increasing  as  each  predictor  variable  is  utilized  to  improve  the  prediction.  When  using  MC  to 
accomplish  ANOVA,  the  presentation  of  "the  source  table"  results  from  the  MC  analysis  would 
be  equivalent  to  those  shown  below. 


Source 

d.f. 

Sums  of  Squares  (SS) 

Mean  Squares  (MS) 

F-ratios 

Effect  A  (A) 

2 

R\  1,2 

SSa/2 

MSa/MSe 

Effect  B  (B) 

2 

R^Y  1,2.3, 4  -  R^Y  1,2 

SSb/2 

MSb/MSe 

Interaction  AB 

4 

R^Y  1„..,8  -  R^Y  1.2,3,4 

SSab/4 

MSab/MSe 

Error  (E) 

N-8-1 

1-R"y1.....8 

SSe/ (N-8-1) 

Total 

N-1 

1.0 

Note:  alternative  F-ratios  for  the  two  main  effects  would  be  MSa/MSab  and  MSb/MSab- 

With  the  MC  approach  which  reports  variance  in  terms  of  z-scores  rather  than  raw-scores, 
the  sums  of  squares  (SSs)  column  can  be  seen,  perhaps  even  more  clearly  than  with  traditional 
ANOVA  source  tables,  to  be  partitioning  the  total  variance  of  the  criterion  into  the  independent 
parts  which  were  caused  by  the  ANOVA  design.  Here  it  can  be  easily  seen  that  the  total  sums  of 
squares  adds  up  to  1.0  which  shows  that  the  total  SSs  for  both  main  effects,  the  interaction 
effects,  and  the  error  term  accounts  for  all  of  the  criterion  variance.  These  SSs  will  be  identical 
to  those  obtained  by  ANOVA  if  the  criterion  variable  scores  had  first  been  converted  to  standard 
scores.  It  should  also  be  noted  that  the  degrees  of  freedom  (dfs)  are  equal  to  the  number  of 
dichotomous  variables  used  to  represent  the  various  effects. 

The  approach  of  using  dichotomous  variables  to  represent  the  levels  of  main  effects  and 
interaction  terms  has  been  covered  here  in  detail  because  this  approach  will  also  be  used  later  in 
the  Rotated  Diagonal  Factors  (RDF)  approach. 
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2.3  Diagonal  Factor  Analysis  (DFA) 

A  matrix  of  correlation  coefficients  shows  the  relationships  among  the  variables 
represented  in  that  matrix.  Each  column  of  the  matrix  represents  a  vector  having  a  variance  of 
one  and  each  row  value  shows  the  relationship  of  one  variable  with  the  column  vector.  These 
vectors,  however,  are  not  necessarily  independent  of  one  another.  Diagonal  Factor  Analysis 
(DFA)  is  a  method  to  obtain  a  set  of  independent  factors  which  contains  all  of  the  same  variance, 
and,  at  the  same  time,  to  obtain  the  relationship  of  each  variable  to  those  independent  factors. 
The  DFA  method  for  doing  this  is  quite  simple.  It  sequentially  derives  successive  independent 
factors  and  removes  its  effect  from  the  matrix.  Each  independent  diagonal  factor  k,  for  example, 
is  found  by  dividing  the  remaining  entry  in  column  k  by  the  square  root  of  value  in  variable  k's 
main  diagonal  cell.  These  values  become  the  diagonal  factor  loadings  (d's)  for  the  first  factor. 
That  is,  the  diagonal  factor  loading  of  variable  i  on  factor  k  will  be: 

di.  =  E,/(rJ-^  . 

The  effect  of  diagonal  factor  k  is  then  removed  from  the  m’‘m  correlation  matrix  using 
the  equation  for  computing  residual  correlations  before  the  next  diagonal  factor  is  found.  The 
equation  for  finding  the  residual  correlation  by  removing  the  effect  of  an  independent  factor, 
which  is  applicable  for  all  rows  and  columns,  is: 

r  =  t*  ^  /HI 

^ijresidual  ij  ^ik  ^jk  * 

The  process  of  finding  diagonal  factors  and  removing  their  effects  is  repeated  until  all 
variables  have  been  selected  and  m  diagonal  factors  have  been  found.  It  should  be  mentioned 
here  that  the  first  diagonal  factor  is  identical  to  the  first  predictor  variable.  The  second  diagonal 
factor  is  equal  to  the  second  predictor  when  all  the  relationships  with  the  first  predictor  has  been 
removed  (symbolized  X2.1).  The  third  diagonal  factor  is  equivalent  to  the  third  predictor  with  all 
the  relationships  attributable  to  the  first  two  predictors  removed  (symbolized  Xs.u),  and  so  forth. 
Thus,  symbolizing  the  ith  diagonal  factors  as  Di,  then  Di  =  Xj.i  i-i . 
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The  above  steps  used  in  accomplishing  DFA  is  actually  one  method  for  accomplishing 
MC.  For  example,  if  one  diagonally  factors  the  predictor  variables  in  a  matrix  containing  m 
predictors  and  a  criterion  variable,  Y,  it  can  be  shown  that 


where: 


R^ 


Y  l.,.m 


d  vxi  +  d 


^  +  ...  + 


,k-l  d  YXin.l,  .  .  .,m-l  5 


. i.J. 

i=l  tom 


2 

R  Y 1.  ..m  =  the  squared  multiple  correlation  for  Y  using  m  predictor  variables, 

d^xi.i . i-i  ^  the  squared  correlation  of  Y  with  X.  with  all  relationships  to 

X.  through  X..J  removed. 


Thus,  these  diagonal  factors  show  the  incremental  increase  in  Y  variance  explained  as 
each  new  predictor  variable  was  selected,  and  each  is  independent  from  all  others.  Thus,  each 
can  also  be  seen  to  be  an  independent  component  of  explained  variance  since  is  the  total 
variance  explained. 


2.3.1  Matrix  Algebra  to  Describe  the  Operations  for  DFA  and  MC 

Matrix  algebra  permits  one  to  indicate  various  kinds  of  matrices.  A  matrix  is  a 
rectangular  set  of  values  having  rows  and  columns.  A  square  matrix  is  one  where  the  number  of 
rows  equals  the  number  of  columns.  A  matrix  is  symbolized  using  brackets  surrounding  a  matrix 
identifier  (e.g.,  [  R  ]  indicates  a  correlation  matrix,  [  F  ]  indicates  a  factor  matrix).  Some  special 
types  of  matrices  are  always  symbolized  in  the  same  way.  For  example,  [  0^  ]  is  referred  to  as  a 
zero  matrix  and  indicates  an  m’‘n  matrix  where  each  cell  value  equals  zero;  [  ]  is  called  an 

identity  matrix  and  always  indicates  an  square  matrix  with  "1"  in  each  of  the  main  diagonal 
(upper  left  to  lower  right)  cells  and  "0"  in  all  other  cells.  Matrix  algebra  can  also  be  used  to 
indicate  various  types  of  operations  on  those  matrices.  Matrices,  if  they  have  the  proper  number 
of  rows  and  columns,  can  be  added  (i.e.,  [A]+[B]),  subtracted  (i.e.,  [A]-[B]),  or  multiplied 
(i.e.,  [A]’‘[B]).  One  matrix  cannot  be  divided  by  another,  but  an  operation  called  matrix 
inversion,  which  is  discussed  in  the  next  subsection,  can  be  done. 
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The  actual  procedure  for  accomplishing  multiple  correlation  (MC)  (i.e.,  when  there  is 
only  one  criterion  variable),  or  multi-criterion  multiple  correlation  (MMC)  (i.e.,  when  there  is 
more  than  one  criterion  variable),  involves  the  same  matrix  operations.  These  operations  can  be 
explained  using  matrix  algebra  terminology.  The  overall  matrix  of  correlations  among  a  set  of  m 
predictor  variables  (X  variables)  and  n  criterion  variables  (Y  variables)  can  be  symbolized  in 
matrix  algebra  as  [  R  ].  However,  [  R  ]  can  also  be  thought  of  as  being  partitioned  into  four 
submatrices,  such  that: 


[  R  ]  =  [  Rxx  ]  [  Rxy  ] 

[  RyX  ]  [  RyY  ]  5 

where: 

[  Rxx  ]  =  the  m^m  correlations  among  the  m  predictor  variables, 

[  Rxy  ]  =  the  m^n  correlations  of  the  m  predictors  with  the  n  criteria, 

[  Ryx  ]  =  the  n*m  correlations  of  the  n  criteria  with  the  m  predictors,  and 

[  Rxx  ]  =  the  n^n  correlations  among  the  n  criterion  variables. 

In  matrix  algebra,  [Ryx]  is  also  known  as  the  transpose  of  [Rxy]  and  can  also  be  symbolized  as 
[R’xy]-  To  transpose  a  matrix,  the  left  to  right  entries  in  each  row  of  the  original  matrix  become 
the  top  to  bottom  entries  in  the  columns  of  the  transposed  matrix.  In  traditional  multiple 
correlation,  there  is  only  one  criterion  variable  (i.e.,  n  =  1).  When  this  is  true,  as  in  ANOVA  and 
MC,  [Rxy]  is  a  single  column  (i.e.,  a  vector)  of  correlations  of  the  predictor  variables  with  the 
criterion  variable,  and  [Ryy]  would  be  a  one-celled  matrix  containing  the  value  1.00  (i.e.,  the 
correlation  of  the  criterion  variable  with  itself).  In  the  following  derivations,  multiple  criteria 
(i.e.,  n  ^  1)  are  permitted  and  this  approach  is  referred  to  as  Multi-Criterion  Multiple  Correlation 
(MMC). 

2.3.2  Matrix  Inversion 


If  the  entries  in  each  row  and  column  of  the  criterion  variables  are  reflected  (i.e.,  all  of 
their  signs  are  changed),  the  top  half  of  the  matrix  [R]  becomes  the  set  of  simultaneous  equations 
that  must  be  solved  to  find  the  best  weights  for  multiplying  the  X  variables  to  predict  the  Y 
variables  (i.e.,  the  weights  to  multiply  the  predictor  z-scores  to  best  predict  the  z-scores  of  the 
criterion  variable(s)  whose  relationships  with  the  predictors  are  shown  in  [Rxy])- 
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One  method  for  solving  this  set  of  simultaneous  equations  is  to  find  what  is  known  as  the 
inverse  of  [Rxx]  whieh  is  symbolized  in  matrix  algebra  as  [R''xx]-  A  technique  for  finding  the 
inverse  of  [Rxx]  is  to  augment  the  predictor  matrix,  [Rxx],  with  two  identity  matrices  and 
an  zero  matrix.  The  m  diagonal  factors  (based  on  the  X  variables)  are  found  and  removed 
from  the  augmented  matrix.  The  original  augmented  matrix,  the  predictor-based  diagonal 
factors,  and  the  residual  matrix  would  be  as  shown  below. 

augmented  Rxx  matrix  diagonal  X  factors  residual  matrix 

[  Rxx  ]  [  Imm  ]  DFA  [  Dxp  ]  and  [  0mm  ]  [  Omm  ] 

[  Imm  ]  [  Omm  ]  yields  [  Dip  ]  [  0mm  ]  [  R  XX  ]  , 

where; 

[  Rxx  ]  =  the  m’^m  correlations  among  the  m  predictor  variables, 

[  Imm  ]  =  an  m’^m  identity  matrix, 

[  Omm  ]  =  an  m’‘m  zero  matrix, 

[  Dxp  ]  =  the  predictors'  correlations  with  (loadings  on)  the  m  diagonal  factors, 

[  Dip  ]  =  the  identity  vectors'  loadings  on  the  m  diagonal  factors,  and 

[  R  'xx  ]  -  the  m’‘m  inverse  of  matrix  [  Rxx  ]• 

If  one  multiplies  the  [R''xx]  matrix  by  the  [Rxy]  matrix,  then  one  obtains  the  z-score 
weights  for  the  multiple  correlation  prediction  equation.  These  z-score  weights  are  traditionally 
referred  to  as  ^-weights.  That  is,  in  matrix  algebra, 

[  Pyx  ]  =  [  R-’xx  ]  X  [  Rxy  ]  , 

where: 

[  pYx  ]  =  the  P-weights  for  the  m  predictors  to  predict  the  n  criteria, 

[  R  'xx  ]  -  the  m^m  inverse  of  matrix  [  Rxx  ],  and 
[  Rxy  ]  =  the  correlations  of  the  m  predictors  with  the  n  criteria. 


2.3.3  Direct  Calculation  of  B-Weights  bv  Matrix  Inversion 


In  the  above  subsection,  it  was  shown  that  DFA  is  part  of  the  procedure  for  calculating 
the  z-score  P-weights  for  either  the  MC  and  the  MMC  methods.  An  interesting  variation  of  this 
procedure  is  to  augment  the  combined  predictor  and  criterion  correlation  matrix  with  identity  and 
zero  matrices  as  shown  below  and  then  perform  the  diagonal  factor  analysis  based  on  the 
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predictor  (i.e.,  X)  variables.  As  shown  below,  the  residual  matrix,  when  this  is  done,  yields  the 
inverse,  the  P-weights  and  the  unexplained  variance  and  covariance  of  the  criterion  (i.e.,  Y) 
variables  (symbolized  here  as  [  Uyy  ])• 


augmented  correlation  matrix  diagonal  factors  the  residual  matrix 


[  Rxx 

] 

[-Rxy 

] 

[  Imm 

] 

[  Dxp  ] 

[  0mm  ] 

[  Omn  ] 

[0mm] 

[-Ryx 

] 

[  Ryy 

] 

[  Onm 

]  yields 

[Dyp] 

[  Onm  ] 

[Uyy] 

[Pvxl 

[  Imm 

] 

[O'nm 

] 

[  Onn 

] 

[  Dip  ] 

[  0mm  ] 

[  P'yx  ] 

[R  'xx] 

where: 

[  Rxx  ]  =  the  correlations  among  the  m  predictor  variables, 

[  -Rxy  ]  =  the  reflected  correlations  of  the  m  predictors  with  the  n  criteria, 

[  -Ryx  ]  =  the  transpose  of  matrix  [  -Rxy], 

[  Ryy  ]  =  the  n’‘n  correlations  among  the  n  criterion  variables, 

[  Imm  ]  -  an  m’^m  identity  matrix, 

[  Onm  ]  =  an  n’^m  zero  matrix, 

[  Omn  ]  =  an  iii’‘n  zero  matrix, 

[  Onn  ]  =  an  n’‘n  zero  matrix, 

[  Dxp  ]  ==  the  predictors'  correlations  with  (loadings  on)  the  diagonal  factors, 

[  Dyp  ]  =  the  criteria's  correlations  with  (loadings  on)  the  diagonal  factors, 

[  Dip  ]  =  the  identity  vectors'  loadings  on  the  diagonal  factors, 

[  Uyy  ]  =  the  n’^n  residual  [  Ryy  ]  matrix  (i.e.,  unexplained  Y  correlations), 

[  Pyx  ]  =  the  beta  weights  for  the  m  predictors  to  predict  the  n  criteria,  and 

[  R  'xx  ]  —  the  ni’‘m  inverse  of  matrix  [  Rxx  ]• 

Submatrix  [  Dyp  ]  contains  the  criterion  variables'  correlations  with  the  independent 
predictor-based  diagonal  factors.  When  the  entries  in  this  submatrix  are  squared,  they  provide 
the  incremental  increase  in  criterion  variance  explained.  The  sums  of  these  squared  correlations 
across  the  factors  for  criterion  j  yield  j's  variance  that  that  can  be  explained  by  the  predictors. 
This  value  is  also  the  squared  multiple  correlation  coefficient  (R^y^)- 

Of  particular  interest  in  having  the  inverse  available  is  that  the  expected  standard 
deviation  of  the  z-score  p-weight  for  predictor  i  to  predict  criterion  variable  j  can  be  found  by  the 
equation: 

SpYjXi  ~  (  ■  IxiXi  (1  -  R^Yj.l  ...  m)  /  (N  -  m  -  1)  )’^  . 
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where  Ixi  xi  is  the  value  found  in  the  ith  diagonal  cell  (i.e.,  row  i  and  column  i)  of  the  inverse 
matrix  [R‘'xx]  and  (1  -  R^Yj.i ...  m)  is  the  value  found  in  the  jth  diagonal  cell  of  the  [Uyy]  matrix. 
The  t-test  of  significance  (with  N-m-1  degrees  of  freedom)  for  each  standard  score  beta  weight  is 
the  ratio  of  the  standard  score  beta  weight  divided  by  the  standard  deviation  for  that  beta  weight. 
That  is, 


P  YjXi  !  • 

While,  on  the  surface,  this  appears  to  be  totally  different  from  traditional  ANOVA 
methods  for  calculating  sums  of  squares,  mean  squares,  model  coefficients,  and  testing  their 
significance,  the  above  calculations  will  arrive  at  mathematically  identical  solutions.  This 
demonstrates  that  data  analysis  by  ANOVA  procedures  is  simply  a  special,  but  limited,  case  of 
the  more  general  MC  or  MR  methods. 

It  should  be  noted  here  that  the  square  of  the  above  equation  will  also  yield  an  F-test 
(df  =  1,  N-m-1)  for  the  standard  score  beta  weight.  That  is, 

+2  _  r2  /t2  2 

t  pYjXi  -  P  YjXi  /  1  Xi  Xi  S  pYj  Xi  • 

FpYjXi  =  (P^YjXi  /  l\i  Xi)  /  ((1  "  R^Yj  1  ...  m)  /  (N  -  m  -  1))  . 

Finally,  it  should  be  noted  that  all  of  the  equations  in  this  subsection  are  equally 
applicable  for  one  or  more  criterion  variables.  Indeed,  the  case  in  which  only  one  criterion 
variable  is  to  be  analyzed  is  simply  a  special  case  of  the  more  general  situation  in  which  several 
criteria  are  analyzed  at  the  same  time. 

2.4  Multivariate  Analysis  of  Variance  (MANOVA) 

Multivariate  ANOVA  (MANOVA)  is  typically  accomplished  when  more  than  one 
criterion  variable  is  to  be  analyzed.  While  the  overall  MANOVA  tests  of  significance  for  main 
effects  and  interactions  is  somewhat  different  than  in  simple  ANOVA,  the  model  coefficients 
produced  for  each  criterion  are  identical  to  those  which  would  have  been  found  had  each 
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criterion  been  separately  analyzed  by  ANOVA.  It  can  also  be  shown  that  the  same  coefficients 
will  be  derived  by  the  matrix  inversion  method  just  described  in  subsection  2.3.2. 

2.5  Common  Factor  Analysis  (FA)  and  Factor  Rotation  (FR) 

Later,  in  discussing  the  Rotated  Diagonal  Factors  (RDF)  approach,  our  interest  will  be  in 
meaningful  independent  factors  that  explain  certain  types  of  variance  which  causes  the  predictors 
to  be  related  to  the  criterion  variables.  Before  discussing  the  RDF  method  for  doing  this,  it  may 
be  helpful  to  discuss  briefly  what  is  referred  to  as  common  factor  analysis  (FA)  and  factor 
rotation  (FR). 


FA  assumes  that  the  total  variance  of  any  variable  is  composed  of  three  unrelated  kinds  of 
variance:  common,  specific,  and  error.  That  is,  for  variable  i, 


The  common  part  of  the  variance  includes  any  part  that  causes  variable  i  to  be  related  to  another 
variable  in  the  correlation  matrix.  The  specific  variance  is  defined  as  non-error  variance  that  is 
unrelated  to  other  variables  in  that  matrix.  Thus,  the  reliable  part  of  the  total  variance  is  the 
common  plus  the  specific.  The  unique  part  of  the  variance  is  the  specific  plus  the  error. 


FA  further  assumes  that  the  common  variance  for  any  variable  in  a  matrix  can,  itself,  be 
composed  of  (i.e.,  explained  by)  K  individual  independent  components.  That  is. 


It  has  been  shown  that  DFA  can  produce  a  set  of  m  independent  diagonal  factors  that  can 
completely  explain  all  of  the  relationships  among  the  m  variables  on  which  those  factors  were 
based.  However,  these  independent  diagonal  factors  explain  all  the  variance  of  the  variables,  not 
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just  the  common  variance.  Further,  and  more  to  the  point,  they  are  not  particularly  meaningful. 
In  fact,  the  values  found  for  these  factors  were  dependent  strictly  on  the  sequence  in  which  the 
diagonal  factors  were  extracted.  Traditional  Factor  Analysis  (FA)  techniques  also  analyze  an 
m’^m  correlation  matrix  [Rxx],  but  for  somewhat  different  purposes.  Typically,  the  investigator 
is  not  particularly  concerned  with  whether  the  variables  contained  in  the  matrix  to  be  factored  are 
predictors  or  criteria.  The  purpose  of  FA  is  to  find  a  smaller  set  (k  <  m)  of  independent 
dimensions  (i.e.,  factors)  which  can  explain  the  correlations  among  all  of  the  variables  in  the 
matrix.  Usually,  K  is  expected  to  be  no  more  than  half  of  what  m  is  (i.e.,  K  ^  m/2).  That  is,  FA 
attempts  to  explain  the  common  variance  in  a  matrix  in  a  parsimonious  fashion.  For  two 
variables  to  be  related  to  each  other,  they  must  share  some  variance  in  common.  Thus,  the  set  of 
factors  with  which  FA  is  concerned  are  referred  to  as  common  factors.  In  fact,  the  goal  of  FA  is 
to  find  an  m’^k  factor  matrix  (i.e.,  [Fxk])  such  that  when  multiplied  by  its  transpose  will  closely 
reproduce  (i.e.,  explain)  all  of  the  off-diagonal  entries  of  the  [Rxx]  matrix. 


The  entry  (f.  J  in  matrix  [Fxk]  for  the  ith  variable  on  factor  k  represents  the  relationship 
of  that  variable  to  that  factor.  These  entries  are  traditionally  referred  to  as  factor  loadings.  The 
explained  correlation  between  any  two  variable  (e.g.,  i  and  j)  is  given  by  the  sum  of  the  products 
of  the  factor  loadings  of  those  two  variables  across  all  of  the  independent  common  factors.  That 
is,  the  explained  correlation  between  variables  i  and  j  (which  is  symbolized  here  as  r'. .)  will  be: 


tf  T  t  =  f  ’‘f  +  f  T  +  +  f  T  +  +  f  T 

k=l 


The  amount  of  common  variance  explained  for  a  variable  by  these  independent  factors  is 
known  as  its  communality  and  is,  traditionally,  symbolized  as  h^.  That  is,  for  variable  i. 


0~  i  c 


(f  *  f  ) 

'■'ik  ‘ik/ 

k=l  to  K 


(to- 

k=l  toK 


The  matrix  which  contains  (a)  the  explained  correlations  in  the  off-diagonal  cells  and 
(b)  the  communalities  in  the  main  diagonal  will  be  referred  to  as  [R  xx]-  The  purpose  of  FA, 
then,  is  to  derive  the  factor  matrix  [Fxk]  such  that,  in  matrix  algebra  notation, 


[  Fxk  ]  ’^  [  F’xk  ]  =  [  Rxx]  • 
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where: 

[  Fxk  ]  =  the  m’^K  common  factor  matrix, 

[  F'xk  ]  =  the  transpose  of  [  Fxk  ],  and 

[  E  XX  ]  -  the  explained  X  variable  correlations  and  communalities. 

Before  going  further,  the  term  eigenvalue  should  be  introduced  and  explained.  It  is  the 
sum  of  the  squares  of  all  of  the  variables'  loadings  on  a  particular  factor.  Remembering  that  a 
factor  loading  is  a  correlation  of  a  variable  to  an  independent  factor,  and  that  the  square  of  a 
correlation  is  the  amount  of  that  variable's  variance  explained  by  that  factor,  then  it  can  be  seen 
that  the  eigenvalue  of  a  factor  is  the  total  amount  of  variance  that  is  being  explained  by  that 
factor  across  all  variables.  The  eigenvalue  of  factor  k  is  symbolized  as  and  is  given  by  the 
equation: 


E.  =  (f\.)  . 

i=  1  to  m 

2.5.2  Extracting  the  Common  Factors 

There  are  several  ways  to  accomplish  FA,  most  of  which  use  mathematical  criteria  to 
determine  both  (a)  the  initial  locations  of  the  independent  factors  and  (b)  when  to  stop  extracting 
factors  (i.e.,  when  the  residual  correlations  probably  represent  only  chance  relationships).  The 
most  frequently  used  FA  technique  are  various  modifications  to  the  Principle  Component  method 
(Hotelling,  1933)  which  maximizes  the  total  common  variance  being  explained  by  each 
successive  factor.  These  modifications,  which  require  estimates  of  communalities  instead  of 
ones  in  the  main  diagonal  of  [  R  xx  ],  are  referred  to  as  the  Principle  Factors  or  Principle  Axes 
method.  The  necessity  for  communalities  for  the  main  diagonal  entries  (instead  of  ones)  is 
because  FA  is  only  interested  in  explaining  common  variance,  rather  than  all  the  variance,  of 
each  variable. 
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When  FA  is  begun,  the  communality  of  each  variable  is  first  estimated  by  various 
techniques  (e.g.,  the  squared  multiple  correlation  of  that  variable  using  all  other  variables  as 
predictors,  the  highest  absolute  correlation  in  that  variable's  row  or  column,  etc.)  Even  though 
these  estimates  may  be  incorrect,  the  FA  proceeds  and  factors  are  extracted  until  a  criterion  for 
stopping  the  factor  extraction  is  met.  New  estimates  of  communalities  are  then  possible  using 
the  latest  factor  loadings  obtained.  The  new  estimates  of  communalities  can  be  substituted  for 
the  old  communality  estimates  and  the  whole  process  can  be  repeated  (i.e.,  iterated),  until  the 
communalities  stabilize  (i.e.,  the  starting  values  of  the  communalities  are  virtually  the  same  as 
the  ending  values). 

2.5.3  Rotating  the  Common  Factors  to  a  Meaningful  Set 

Because  various  mathematical  criteria  were  used  in  the  factor  extraction  process,  the  final 
independent  common  factors  will  rarely  represent  dimensions  which  are  particularly  useful  in 
understanding  why  the  variables  correlated  the  way  they  did.  However,  the  extracted  factors  can 
usually  be  rotated  to  a  more  meaningful  set  of  independent  dimensions.  By  a  meaningful  set  of 
factors,  it  is  meant  only  that  the  rotated  factors  should  have  loadings  (i.e.,  correlations  of  the 
variables  with  those  factors)  that  are  useful  to  the  investigator  in  explaining  the  nature  of  the 
factors.  For  example,  it  is  particularly  useful  if  some  variables  have  high  loadings  on  some 
factors  but  do  not  load  on  any  other  factors.  In  that  circumstance,  the  investigator  may  be  able  to 
identify  what  it  is  that  those  particular  variables  have  in  common  that  the  other  variables  do  not 
possess.  In  that  way,  the  nature  of  that  factor  can  be  deduced.  Factor  Rotation  (FR)  of 
independent  factors  can  be  accomplished  either  graphically  or  mathematically  and  does  not 
change  the  ability  of  the  new  set  of  factors  to  explain  the  correlations  among  the  variables.  The 
Varimax  technique  (Kaiser,  1959)  is  a  frequently  used  rotation  technique  that  uses  mathematical 
criteria  to  search  for  a  set  of  factors  having  what  is  referred  to  as  simple  structure  (i.e.,  each 
factor  has  two  or  more  variables  with  high  loadings  on  it  and  all  other  variables  have  near-zero 
loadings  on  it). 
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2.6  Canonical  Analysis  (CA) 

Before  discussing  the  RDF  method,  a  brief  discussion  of  Canonical  Analysis  (CA)  will  be 
given  because  it  has  certain  similarities  to  both  RDF  and  to  MANOVA.  CA  was  also  devised  by 
Hotelling  (Hotelling,  1935).  Like  FA,  CA  is  concerned  with  common  variance,  but  its  interest  is 
limited  to  explaining  the  common  variance  shared  by  two  sets  of  variables,  but  not  the  additional 
common  variance  that  may  be  shared  by  variables  within  either  set.  Of  importance  to  this 
discussion  is  the  conclusion  (in  the  discussion  of  MC)  that  both  subject  effects  and  experimental 
effects,  when  coded  as  dichotomous  variables,  can  be  considered  to  be  one  set  of  variables 
(i.e.,  the  predictor  set),  while  multiple  criteria  can  be  considered  as  a  second  set  of  variables 
(i.e.,  the  criterion  set).  CA  is  generally  applicable  to  any  situation  in  which  one  set  of  X  variables 

is  related  to  another  set  of  Y  variables,  and  the  scores  for  both  sets  of  variables  are  available  for  a 

common  group  of  cases  (e.g.,  individuals).  As  before,  if  the  m  variables  in  set  X  and  the  n 
variables  in  set  Y  are  intercorrelated,  then  the  overall  intercorrelation  matrix  can  be  partitioned 
into  four  submatrices  such  that: 

[  R  ]  =  [  Rxx  ]  [  Rxy  ] 

[  Ryx  ]  [  Ryy  ]  , 

where  submatrices  [Rxx],  [Rxy],  [Ryx],  and  [Ryy]  contain  the  same  variables  as  discussed 
earlier. 

2.6.1  Early  Canonical  Techniques 

Hotelling's  original  canonical  method  related  the  two  sets  of  scores  by  finding  a  pair  of 
vectors  (one  from  each  set)  that  correlated  higher  than  any  other  pair  of  vectors.  The  effects  of 
the  best  pair  of  vectors  were  then  removed  (as  were  the  diagonal  factors  in  DFA  and  MC)  from 
the  [R]  matrix.  Then  a  second  pair  of  vectors  were  obtained  that,  together,  yielded  the  highest 
correlation  and  were  orthogonal  (i.e.,  unrelated)  to  the  first  pair  of  vectors.  This  procedure  was 
continued  until  there  were  as  many  pairs  of  vectors  as  there  were  variables  in  the  smaller  set. 
This  procedure  was  actually  started  by  finding  the  ^-weights  (betas)  for  predicting  the  variables 
in  one  set  from  the  variables  in  the  other  set.  These  P-weights  for  predicting  set  Y  variables  from 
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the  set  X  variables  were  found  by  the  equation  which  was  discussed  earlier.  That  is,  [  Pyx  ]  = 
[R''xx]  X  [  Rxy  ]  •  Similarly,  the  P- weights  for  predicting  the  X  set  variables  from  the  Y  set 
variables  were  found  by  a  complementary  equation.  That  is,  [  Pxy  ]  ~  [R  'yy]  X  [Ryx]  •  Next, 
the  cross  products  of  the  two  sets  of  P-weights  were  obtained,  using  either  [Cxx]  =  [Pxy]  x  [Pyx] 
(if  m  is  smaller)  or  [Cyy]  =  [Pyx]  x  [Pxy]  (if  n  is  smaller).  Next,  the  eigenvalue  (i.e.,  the 
proportion  of  variance  explained  by  that  pair  of  vectors)  of  the  appropriate  [C]  matrix  was 
computed.  The  square  root  of  the  eigenvalue  was  referred  to  as  the  canonical  correlation  Rci, 
where  i  represents  the  ith  pair  of  vectors  found  as  explained  above.  Rci  can  be  tested  for 
significance  using  a  value  based  on  the  eigenvalue. 

2.6.2  The  Reduced  Matrix  Canonical  (RMCt  Approach 

While  the  traditional  Hotelling  CA  technique  was  useful  in  determining  the  amount  and 
significance  of  overlapping  common  variance  between  two  sets  of  variables,  interpretation  of  the 
successive  sets  of  weights  for  each  of  the  independent  canonical  factors  was  rarely  possible. 
Hotelling's  original  method  did  not  obtain  factor  loadings,  but  simply  the  beta  weights  for  the 
variables  on  those  factors.  Further,  the  p-weights  were  for  independent  factors  that  had  never 
been  rotated  to  meaningful  positions.  Under  such  conditions,  interpreting  the  canonical  factors 
was  next  to  impossible.  This  author  developed  a  modified  approach  (Wherry,  Jr.,  1975)  for 
canonical  correlation  that  permitted  the  obtaining  of  rotatable  orthogonal  factor  loadings  from  the 
factor  space  shared  by  the  two  sets  of  variables.  This  approach,  referred  to  as  the  Reduced 
Matrix  Canonical  (RMC)  approach,  was  first  used  on  a  method  he  had  developed  (Wherry,  Jr., 
1965)  for  the  K-Coefficient,  a  Pearson-type  substitute  for  the  Contingency  Coefficient.  The 
RMC  approach  had  an  added  advantage  over  traditional  CA  at  that  time  in  that  each  successive 
extracted  factor  would  explain  the  maximum  amount  of  common  criterion  variance.  When  all 
possible  canonical  factors  had  been  extracted,  both  approaches  would  explain  the  same  amount 
of  criterion  variance  across  all  variables.  However,  it  is  often  the  case  that  all  canonical  factors 
are  not  significant.  For  the  same  number  of  canonical  factors,  the  RMC  approach  will  always 
explain  as  much  or  more  of  the  common  criterion  variance. 
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The  approach  used  in  the  RMC  approach  was  to,  first,  diagonally  factor  the  larger  set  of 
variables.  For  discussion  purposes,  suppose  there  are  more  X  than  Y  variables  (i.e.,  m  >  n). 
When  all  m  of  the  predictor-based  diagonal  factors  have  been  removed  from  both  matrices,  the 
diagonal  factors  and  residual  correlation  matrix  will  be  as  shown  below. 


where: 


original  correlations 

diagonal  factors 

residual  matrix 

[  Rxx 

] 

[  Rxy  ] 

RDF 

[  Dxm  ]  and 

[  0mm 

]  [  Omn  ] 

[  Ryx 

] 

[  Ryy  ] 

yields 

[  Dym  ] 

[  Onm 

]  [  Uyy  ]  , 

[  Rxx 

], 

[  Rxy], 

[  Ryx], 

and  [  Ryy  ] 

are  as  discussed  earlier. 

[  Dxp  ]  -  the  loadings  of  predictors  on  the  predictor-based  diagonal  factors, 

[  Dyp  ]  =  the  loadings  of  criteria  on  the  predictor-based  diagonal  factors, 

[  0mm  ]  =  an  m’^in  zero  matrix  =  residual  correlations  among  the  predictors, 

[  Omn  ]  ~  an  m’‘n  zero  matrix  =  residual  correlations  of  criteria  and  predictors, 
[  Onm  ]  =  an  n^m  zero  matrix  =  residual  correlations  of  criteria  and  predictors, 
[  Uyy  ]  ~  the  n’‘ii  residual  correlations  among  the  n  Y  variables. 


Any  remaining  values  in  matrix  [  Uyy  ]  cannot  be  attributable  to  common  variance  shared  by  the 
two  sets  of  variables  because  the  diagonal  factors  already  explain  all  of  the  variance  in  the  larger 
X  set.  Thus,  these  residual  correlations  may  be  ignored.  However,  the  loadings  of  the  large;  and 
smaller  sets  of  variables  on  the  predictor-based  independent  diagonal  factors  must  explain  all  of 
the  interrelationships  among  the  two  sets  of  variables,  all  of  the  variance  among  the  larger  X  set, 
and  some  of  the  variance  of  the  Y  set.  The  correlations  and  communalities  of  the  smaller  Y  set 
of  variables  (in  the  space  occupied  by  the  larger  X  set)  may  now  be  found  by  the  equation 

rvxiYxj  =  (dyi.  J  >  or,  in  matrix  algebra  terms, 

m=l 


where: 


[Ryxyx]  -  [  Dyp  ]  [  D'yp  ]  , 

[  Ryxyx  ]  ==  the  criteria  correlations  explained  by  predictor-based  diagonal 
factors, 

[  Dyp  ]  =  the  criteria  eorrelations  with  the  predictor-based  diagonal  factors, 
and 

[  D'yp  ]  =  the  transpose  of  matrix  [  Dyp  ]. 
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Using  this  new  [  Ryxyx  ]  matrix,  which  contains  the  correlations  of  the  Y  variables  in  the 
X  space,  a  traditional  Principle  Factors  Analysis  (PFA)  is  then  accomplished.  This  yields 
independent  common  factors  which  explain  as  much  of  this  criterion  variance  as  possible  with 
each  successive  factor.  Subsequent  steps  in  the  RMC  approach  derive  the  loadings  of  the  X 
variables  on  the  derived  common  factors,  and  it  is  this  set  of  factors  which  can  be  rotated  to  find 
a  meaningful  set.  Again,  the  use  of  DFA  played  a  central  part  in  CA  in  the  RMC  approach. 

2.7  The  Rotated  Diagonal  Factor  (RDF) ^Approach 

In  the  Introduction  (Section  1),  it  was  stated  that  "in  a  multi-task,  multi-criterion  study, 
the  total  variance  in  performance  on  each  criterion  should  be  able  to  be  divided  into  three  types 
of  independent  variance  components:  (a)  between-tasks,  (b)  within-tasks  (but  between  criteria), 
and  (c)  within-criterion."  The  Rotated  Diagonal  Factors  (RDF)  approach,  to  be  discussed  now, 
permits  accomplishment  of  this  objective. 

It  has  been  shown  that  MANOVA  and  MMC  techniques  permit  derivation  of  model 
coefficients  for  predicting  any  criterion  variable.  In  both  techniques,  the  analysis  is  concerned,  in 
one  way  or  another,  with  all  of  the  criterion  variance,  some  of  which  is  attributable  to  main 
effects,  some  to  interaction  effects,  and  some  to  the  error  term.  It  has  also  been  seen  that  both  FA 
and  CA  can  be  used  to  derive  independent  dimensions  of  shared  common  variance,  but  neither  of 
these  latter  techniques  will  necessarily  account  for  all  of  the  criterion  variance.  Thus,  FA  and  CA 
do  not  accomplish  what  is  needed. 

The  RDF  approach,  however,  will  derive  independent  dimensions  that  can  both  explain 
all  of  the  variance  of  the  criterion  variables  and  all  of  the  covariance  of  the  criterion  variables 
with  the  predictor  variables.  The  RDF  approach  consists  of  four  major  steps: 

1 .  obtaining  the  criterion-based  diagonal  factors, 

2.  rotating  these  factors  to  a  meaningful  structure, 

3.  using  the  predictor  variables  with  MMC  to  predict  the  rotated  diagonal  factors,  and 

4.  determining  and  testing  the  predictor  B-weights  (i.e.,  model  coefficients). 

These  four  steps  are  explained  in  more  detail  in  the  following  subsections. 
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2.7.1  Step  1 :  Obtaining  the  Criterion-Based  Diagonal  Factors 


The  initial  step  in  the  RDF  approach  is  to  obtain  a  set  of  diagonal  factors.  However, 
unlike  MC  or  MMC,  this  set  of  factors  is  based  on  diagonally  factoring  the  criterion  set  of 
variables  rather  than  the  predictor  set.  Diagonally  factoring  the  combined  predictor  and  criterion 
matrix  obtains  loadings  of  both  predictor  and  criterion  variables  on  all  m  of  the  criterion-based 
diagonal  factors.  That  is, 

original  correlations 

[  Rxx  ]  [  Rxy  ]  DFA 

[  Ryx  ]  [  Ryy  ]  yields 

where: 

[  Rxx  ]  is  the  m’‘ni  correlation  matrix  of  the  predictor  variables, 

[  Rxy  ]  is  the  m^'n  correlation  matrix  of  the  predictors  with  the  criteria, 

[  Ryx  ]  is  the  n’‘ni  correlation  matrix  of  the  criteria  with  the  predictors, 

[  Ryy  ]  is  the  n’^n  correlations  matrix  of  the  criterion  variables, 

[  Dxc  ]  is  the  m’^n  criterion-based  diagonal  factor  matrix  (the  X  set  loadings), 

[  Dyc  ]  is  the  n^n  criterion-based  diagonal  factor  matrix  (the  Y  set  loadings), 

[  Uxx  ]  is  the  residual  correlation  matrix  of  the  predictor  variables, 

[  OxY  ]  is  an  m^n  zero  matrix  =  residual  correlations  of  predictors  and  criteria, 

[  Oyx  ]  is  an  n’‘m  zero  matrix  =  residual  correlations  of  criteria  and  predictors, 

[  OxY  ]  is  an  n^n  zero  matrix  =  residual  correlations  of  the  criterion  variables. 


diagonal  factors 

[  Dxc  ] 

[  Dyc  ]  and 


the  residual  matrix 
[  Uxx  ]  [  OxY  ] 

[  Oyx  ]  [  Oyy  ]  , 


These  diagonal  factors  account  for  (i.e.,  explain)  all  of  the  criterion  variance,  all  of  the 
correlations  among  the  criterion  variables,  and  all  of  the  correlations  of  the  predictor  variables 
with  the  eriterion  variables.  Because  of  this  property,  the  criterion  variables'  communalities 
across  the  n  factors  will  all  equal  1.0.  Thus,  like  in  MANOVA,  all  of  the  criterion  variance  is 
explained. 
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2.7.2  Step  2:  Rotating  the  Criterion-Based  Diagonal  Factors 


The  second  step  in  the  RDF  approach  is  to  rotate  these  criterion-based  diagonal  factors  to 
a  meaningful  structure.  Because  these  diagonal  factors  are  orthogonal  (i.e.,  mathematically 
independent),  they,  like  any  set  of  common  factors,  can  be  rotated  to  find  a  more  meaningful  set 
of  factors.  To  be  successful,  the  rotation  of  the  criterion-based  diagonal  factors  must  result  in  a 
structure  that  contains  factors  that  can  be  associated  with  the  three  desired  types  mentioned 
earlier: 


•  between-tasks  factors:  significant  loadings  on  criteria  from  more  than  one  task, 

•  within-task  factors:  significant  loadings  on  more  than  one  criterion  for  one  task 
but  no  significant  loadings  on  criteria  from  any  other  task,  and 

•  within-criterion  factors:  only  one  significant  loading  on  one  criterion  of  one  task. 

It  should  be  noted  here  that  the  first  two  types  of  factors  (i.e.,  between-tasks  and  within-task)  do 
account  for  common  criterion  variance  while  the  final  type  (i.e.,  within-criterion)  only  accounts 
for  unique  criterion  variance.  Thus,  the  RDF  approach  accounts  for  all  of  the  criterion  variance, 
but  it  also  separates  the  common  from  the  unique  variance.  Such  a  set  of  factors  cannot  possess 
simple  structure  and,  because  of  this,  the  Varimax  technique  cannot  be  used  to  accomplish  the 
needed  rotations.  However,  the  factors  can  be  rotated  graphically  to  find  such  factors.  The  final 
rotated  factor  loadings  will  be  referred  to  as  shown  below. 


diagonal  factors  rotated  diagonal  factors 

[  Dxc  ]  factor  rotation  [  Fxc  ] 

[  Dyc  ]  yields  [  Fyc  ]  • 

where: 


[  Dxc  ] 
[  Dyc  ] 
[  Fxc  ] 
[  Fyc  ] 


predictors'  correlations  on  unrotated  criterion-based  diagonal  factors, 
criteria's  correlations  on  unrotated  criterion-based  diagonal  factors, 
predictor's  correlations  on  the  rotated  diagonal  factors,  and 
criteria's  correlations  on  the  rotated  diagonal  factors. 
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2.7.3  Step  3:  Predicting  the  RDFs  Using  Predictor  Variables 


The  third  step  in  the  RDF  approach  is  to  employ  the  multi-criterion  multiple  correlation 
(MMC)  technique  and  use  the  X-variables  to  predict  the  rotated  diagonal  factors.  The 
correlations  among  the  predictors,  [  Rxx],  are  already  known.  The  correlations  of  the  predictors 
with  the  rotated  diagonal  factors,  [  Fxc  ],  were  found  in  the  previous  step.  The  correlations 
among  the  rotated  diagonal  factors,  [  Rcc],  are,  because  they  are  independent  factors,  an  m’^ni 
identity  matrix.  The  method  for  augmenting-a  combined  matrix  of  predictors  and  criteria  for 
accomplishing  the  MMC  was  described  earlier  in  subsection  2.3.3  which  dealt  with  the  direct 
calculation  of  p-weights  by  matrix  inversion.  Here,  the  criteria  to  be  predicted  are  now  the 
rotated  diagonal  factors.  The  initial  augmented  matrix,  predictor-based  diagonal  factors,  and 
residual  matrix  after  diagonally  factoring  are: 


where: 


the  initial  augmented  matrix 

[  Rxx  ]  [-Fxc  ]  [  Imm  ] 
[-F'xc]  [Rcc]  [Onn,] 

[  Imm  ]  [  Omn  ]  [  0  mm  ] 


diagonal  factors  the  final  residual  matrix 

[  Dxp  ]  [  0mm  ]  [  Omn  ]  [  Omni] 

[  DcP  ]  [  Onm  ]  [  Ucc  ]  [  Pcx] 

[  Dip  ]  [  0mm  ]  [  P'cx  ]  [R’'xx], 


[  Rxx  ]  =  the  original  m^m  correlations  among  the  predictors, 

[  -Fxc  ]  =  the  reflected  predictor  correlations  with  the  rotated  diagonal  factors, 
[  -F'xc  ]  =  the  transpose  of  [  -Fxc  ], 

[  Rcc  ]  =  the  correlations  of  the  rotated  diagonal  factors,  an  identity  matrix, 

[  Imm  ]  =  an  m’‘m  identity  matrix, 

[  Onm  ]  =  an  n’^m  zero  matrix, 

[  Omn  ]  =  an  m’^n  zero  matrix, 

[  0mm  ]  =  an  m’^m  zero  matrix, 

[  Dxp  ]  =  the  predictors'  loadings  on  the  predictor-based  diagonal  factors, 

[  Dcp  ]  =  the  RDFs'  loadings  on  the  predictor-based  diagonal  factors, 

[  Dip  ]  =  the  identity  vector  loadings  on  the  predictor-based  diagonal  factors, 

[  Ucc  ]  =  the  n’‘n  unexplained  variance  of  the  rotated  diagonal  factors, 

[  Pcx  ]  =  the  n’"m  betas  to  predict  rotated  diagonal  factors  using  predictors, 

[  P'cx  ]  =  the  m’^n  transpose  of  [  Pcx  ],  and 

[  R  'xx  ]  =  the  m’^m  inverse  of  [  Rxx]- 
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Submatrix  [Dcp]  contains  the  correlations  of  the  rotated  diagonal  factors  with  the 
predictor-based  diagonal  factors.  When  these  entries  are  squared  and  summed  across  a  given 
rotated  diagonal  factor,  they  show  the  total  percentage  of  that  variance  of  that  RDF  explained  by 
the  predictors.  This  is  its  squared  multiple  correlation  coefficient.  Each  entry  squared  is  the 
incremental  increase  in  the  rotated  diagonal  factors'  variance  explained  by  the  successive 
predictor  variables.  If  the  separate  sums  of  these  entries  are  obtained  for  the  predictor  variables 
associated  with  each  main  effect  and  each  interaction  term,  then  the  total  variance  explained  by 
each  effect  can  be  shown  in  the  "sums  of  squares"  column  of  a  "source  table".  When  those 
values  are  divided  by  their  appropriate  degrees  of  freedom,  the  resulting  values  can  be  shown  in 
the  "mean  squares"  column,  and  the  mean  squares  can  then  be  used  to  test  the  significance  of  the 
various  effects. 

2.7.4  Step  4:  Obtaining  the  B- Weights  for  the  X  Variables 

The  final  step  in  the  RDF  approach  is  computation  of  the  raw-score  weights  to  apply  to 
the  X  variables  for  predicting  the  rotated  diagonal  factors.  Submatrix  [  Pcx  ]  provides  the  z-score 
beta  weights  for  the  predictors.  To  convert  from  z-score  p-weights  to  raw-score  B-weights,  the 
means  and  standard  deviations  of  the  rotated  diagonal  factors  must  be  known.  There  are  no 
actual  raw  scores  for  the  rotated  diagonal  factors,  and  any  desired  values  may  be  assigned  for  that 
purpose.  By  assigning  means  of  zero  and  standard  deviations  of  one  for  each  RDF,  the  derived 
B-weights  take  on  interesting  properties.  It  should  be  remembered  that  all  of  the  predictor 
variables  are  dichotomous  variables  with  raw  scores  of  "1"  (if  that  level  was  applicable)  or  "0"  (if 
that  level  was  inapplicable).  Thus,  while  B-weights  for  all  predictor  variables  may  be  available, 
many  of  them  will  be  multiplied  by  zero  for  the  prediction  of  a  specific  data  case.  Indeed,  the 
only  B-weights  that  will  contribute  to  the  prediction  of  a  specific  data  case  will  be,  at  most,  one 
from  each  level  of  main  effect  or  interaction.  Because  of  this  property,  each  B-weight  describes, 
in  terms  of  standard  deviation  units,  the  impact  of  that  specific  level  of  that  specific  effect  for 
that  RDF. 
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In  this  section,  various  multivariate  analyses  have  been  discussed  and  compared.  The 
purposes,  strengths,  and  weaknesses  of  these  different  techniques  have  also  been  discussed  in 
relationship  to  the  RDF  approach.  It  has  been  shown  that  the  RDF  approach  incorporates  and 
combines  logic  from  Multivariate  Multiple  Correlation  (MMC),  Multivariate  Analysis  of 
Variance  (MANOVA),  Factor  Analysis  (FA),  and  Factor  Rotation  (FR)  to  provide  a  new  and 
different  approach  to  the  analysis  of  studies  involving  multiple  tasks  and  multiple  criteria  for 
each  task.  In  the  next  section,  a  practical  example  of  the  use  of  the  RDF  approach  will  be 
demonstrated. 
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3.  DEMONSTRATION  OF  THE  RDF  APPROACH 

3.1  The  Experimental  Study 

The  experimental  study  used  to  demonstrate  the  RDF  approach  was  conducted  at  the 
Naval  Air  Warfare  Center  in  Warminster,  Pennsylvania,  in  1992.  Its  major  purpose  was  in 
identifying  the  impact  of  temporarily  automating  one  of  three  typical  ongoing  Naval  fighter  pilot 
tasks.  The  three  tasks  included:  (a)  a  continuous  tracking  (Trk)  task,  (b)  a  discrete  tactical 
assessment  (TA)  task,  and  (c)  a  discrete  communications  (Com)  task.  These  tasks  are  discussed 
below. 

3.1.1  The  Tracking  Task 

The  tracking  (Trk)  task  was  a  first-order  compensatory-type  tracking  task  in  which  Ss 
attempted  to  null  out  error  in  both  elevation  and  azimuth  by  manipulation  of  a  control  stick.  Four 
measures  of  performance  used  to  evaluate  tracking  performance  on  this  task  included:  (a)  RMS 
Error  in  Elevation,  (b)  RMS  Error  in  Azimuth,  (c)  Stick  Manipulations  (i.e.,  whenever  S 
manipulated  the  stick  in  a  different  direction  or  changed  significantly  the  amount  of  stick 
deviation)  in  Elevation,  and  (d)  Stick  Manipulations  in  Azimuth.  Two  additional  tracking 
performance  measures  (i.e..  Total  RMS  Error  and  Total  Stick  Manipulations)  were  available,  but 
were  not  used  in  the  RDF  approach.  The  decision  not  to  use  these  measures  was  based  on  the 
fact  that  unique  variance  in  any  of  the  first  four  measures  would  become  common  variance  in  a 
composite  of  those  measures.  Using  both  composite  variables  and  their  individual  variables  in 
the  same  intercorrelation  matrix  is  never  recommended  if  factor  analysis  is  to  be  accomplished 
on  that  matrix  since  it  necessarily  confuses  common  and  unique  variance. 
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3.1.2  The  Tactical  Assessment  Task 

The  tactical  assessment  task  (TA)  consisted  of  a  binary  classification  (e.g.,  hostile  or 
non-hostile)  of  new  targets  that  appeared  on  the  Ss’  tactical  screen  at  the  time  that  they  came 
within  a  certain  range  of  the  target.  Ss  reported  their  assessments  of  the  target  by  pressing  either 
a  "hostile"  or  "non-hostile"  button  for  the  classification  of  the  target.  Measures  of  performance 
available  for  this  task  included;  (a)  Percent  of  Correct  Responses,  (b)  Median  Time  for  Correct 
Responses,  (c)  Percent  of  Incorrect  Responses,  (d)  Median  Time  for  Incorrect  Responses,  and 
(e)  Percent  of  Misses  (i.e.,  no  response  in  a  specified  period  of  time).  Since  (e),  the  criterion 
measure  of  percent  of  misses,  is  completely  predictable  from  knowing  (a)  and  (c),  its  inclusion 
would  not  explain  any  additional  criterion  variance,  therefore,  it  was  not  included  in  the  analysis. 

3.1.3  The  Communication  Task 

A  communication  (Com)  task  consisted  of  verbal  messages  that  contained  three  parts. 
Each  message  began  with  (a)  a  call  sign  followed  by  (b)  a  command  to  change  a  parameter 
(either  altitude  or  heading)  and,  finally,  (e)  the  value  to  which  the  specified  parameter  was  to  be 
changed.  For  example,  the  message  communicated  might  be,  "Bantam,  change  heading  to 
180  degrees."  Ss  were  not  to  respond  in  any  fashion  unless  the  call  sign  was  the  specific  one  they 
had  been  assigned.  This  was  ineluded  as  part  of  the  task  because  pilots  typically  hear  commands 
directed  to  others,  and  must  mentally  screen  out  those  types  of  messages.  The  method  of 
response  was  either  verbal  (for  some  Ss)  or  manual  (for  all  other  Ss).  If  a  S's  mode  of  response 
was  verbal,  he  had  been  instructed  to  respond  either,  "Roger,  heading"  or  Roger,  altitude."  If  the 
S's  mode  of  response  was  manual,  he  had  been  instructed  to  manipulate  a  two-position  switch 
mounted  on  the  control  stick  to  its  first  (altitude)  or  second  (heading)  position.  Measures  of 
performanee  on  the  Com  task  included:  (a)  Percent  of  Correct  Responses,  (b)  Median  Time  for 
Correct  Responses,  (c)  Percent  of  Incorrect  Responses,  (d)  Median  Time  for  Incorrect  Responses, 
and  (e)  Percent  of  Misses  (i.e.,  no  response  in  a  specified  period  of  time).  Results  indicated  no 
actual  misses  occurred,  therefore,  that  measure  was  not  used.  Since  there  were  no  misses, 
measure  (e)  was  completely  predictable  from  knowing  (a).  Thus,  measure  (c)  was  also  excluded 
for  that  reason. 
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3.1.4  Experimental  Manipulations 


Experimental  manipulations,  in  addition  to  the  assignment  of  various  Ss  to  either  the 
Verbal-Response  or  Manual-Response  mode  groups  for  the  Com  task,  consisted  of  Conditions, 
Order  (of  conditions),  and  Period,  each  of  which  will  be  clarified  below.  The  Order  that  Ss 
performed  the  Conditions  was  randomized  by  the  investigator.  Seven  experimental  conditions 
were  investigated.  There  were  seven  performance  sessions,  each  of  which  consisted  of  three 
consecutive  performance  periods.  A  short  break  followed  the  conclusion  of  each  session 
(i.e.,  after  Period  3  was  finished).  During  both  the  first  and  third  periods  of  these  sessions,  Ss 
performed  all  three  of  the  tasks  discussed  above,  and  all  three  tasks  were  at  what  was  judged  to 
be  an  "easy"  level  of  difficulty.  Conditions  referred  to  possible  modifications  to  the  tracking  or 
tactical  assessment  tasks  during  period  2.  Such  modifications  included; 


•  changing  TA  or  Trk  to  a  more  difficult  level,  and/or 

•  automating  TA  or  Trk  so  the  Ss  did  not  have  to  perform  it. 

The  Com  task  was  always  at  its  "easy"  level  and  was  never  automated.  The  seven  experimental 
conditions  were  thus  predicated  on  whether  the  tracking  or  tactical  assessment  tasks  were  "easy" 
or  "hard"  and  whether  they  were  automated  or  not  during  period  2.  The  seven  different 
conditions  presented  to  each  S  during  some  period  2  are  as  shown  below. 


Condition 

Communications 

Tracking 

Tactical  Assessment 

1 

easy 

easy 

easy 

2 

easy 

easy 

hard 

3 

easy 

hard 

easy 

4 

easy 

easy 

hard  (automated) 

5 

easy 

hard  (automated) 

easy 

6 

easy 

easy  (automated) 

hard 

7 

easy 

hard 

easy  (automated) 
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3.1.5  Subjects 

Originally,  an  equal  number  of  Ss  were  planned  for  the  verbal-response  and  manual- 
response  mode  groups  for  the  Com  task.  However,  because  of  missing  data  for  some  conditions 
and  some  tasks,  a  total  of  eight  Ss  had  complete  data  for  the  verbal-response  Com  mode  group 
while  six  Ss  had  complete  data  for  the  manual-response  Com  mode  group. 

3.1.6  Creating  the  Dichotomous  Predictor  Variables 


The  main  objective  of  this  study  was  to  determine  the  residual  effects  of  the  presented 
conditions.  That  is,  the  analysis  attempted  to  determine  if,  having  been  exposed  to  the  various 
conditions,  the  Ss  would  behave  differently  in  Period  3  than  they  had  behaved  during  Period  1. 
Because  of  this,  only  performance  data  from  Period  1  and  Period  3  were  included  for  analysis. 
For  each  criterion,  each  of  the  14  Ss  had  scores  on  the  seven  conditions  for  the  two  periods  of 
interest  (i.e.,  Period  1  and  Period  3).  Thus,  the  total  number  of  data  cases  (N)  for  each  criterion 
was  196  (=  14  X  7  X  2).  The  major  experimental  effects  studied  and  their  accompanying  degrees 
of  freedom  are  shown  below: 


Experimental  and  Individual  Differences  Effects _ Degrees  of  Freedom 


G 

2  Groups  of  Com  task  response-modes  (Verbal  or  Manual) 

1 

=  (2-1) 

S/V 

8  Ss  within  Group  1  (the  verbal-response  mode) 

7 

=  (8-1) 

S/M 

6  Ss  within  Group  2  (the  manual-response  mode) 

5 

=  (6-1) 

0 

7  Orders  of  performing  the  conditions 

6 

=  (7-1) 

P 

2  Periods  of  performance  (Period  1  and  Period  3) 

1 

=  (2-1) 

c 

7  Conditions 

6 

=  (7-1) 

PxC 

2  Periods  by  7  Conditions 

6 

=  (2-l)(7-]) 

PxO 

2  Periods  by  7  Orders 

_6 

=  (2-l)(7-l) 

Effects  degrees  of  freedom 

38 

Residual  degrees  of  freedom 

157 

Total  degrees  of  freedom 

195 

=  (N-1) 

Because  the  order  of  conditions  was  assigned  randomly,  some  combinations  of  condition 
and  order  (CxO)  did  not  occur  at  all  while  other  combinations  of  CxO  occurred  by  chance  alone 
six  times.  Because  of  missing  data  for  some  CxO  cells  and  the  widely  different  number  of  cases 
in  other  CxO  cells,  it  was  decided  not  to  attempt  analysis  of  the  CxO  interaction. 
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3.2  Data  Analyses 


Dichotomous  predictor  variables  (coded  as  "1"  if  applicable  to  a  data  case  and  "0"  if 
inapplicable)  were  created  for  each  potential  effect's  degrees  of  freedom  as  discussed  above. 
Data  analyses  following  the  procedures  described  in  subsection  2.7  were  carried  out  on  a 
Macintosh  SE  computer  using  "The  RDF/MMC  Analysis  Program"  developed  by  the  author 
(Wherry,  Jr.,  1994).  The  program  can  perform  the  RDF  analysis  or  an  MMC.  Results  from  this 
program  were  checked  for  accuracy  against  commercially  available  statistical  packages  offering 
MMC  analysis.  It  yielded  identical  results  and  tests  of  significance. 


Table  1.  Means  and  Standard  Deviations  of  Predictor  and  Criterion  Variables 


Vb# 

Variable  Name 

Mean 

St.Dev. 

Vb# 

Variable  Name 

Mean 

St.Dev. 

Groups  (of  Subjects) 

Period  x  Condition  Interactions 

XOl 

Com  Verbal-Response 

0.5714 

0.4949 

X27 

P  1  xC  1 

0.0714 

0.2575 

Ss  (in  Verbal  Response  Group) 

X28 

P  1  xC2 

0.0714 

0.2575 

X02 

Subject  VI 

0.0714 

0.2575 

X29 

P  1  xC3 

0.0714 

0.2575 

X03 

Subject  V2 

0.0714 

0.2575 

X30 

P  1  xC4 

0.0714 

0.2575 

X04 

Subject  V3 

0.0714 

0.2575 

X31 

P  1  X  C  5 

0.0714 

0.2575 

X05 

Subject  V4 

0.0714 

0.2575 

X32 

P  1  xC6 

0.0714 

0.2575 

X06 

Subject  V5 

0.0714 

0.2575 

Period  x  Order  Interactions 

X07 

Subject  V6 

0.0714 

0.2575 

X33 

P  1  xO  1 

0.0714 

0.2575 

X08 

Subject  V7 

0.0714 

0.2575 

X34 

P  1  x02 

0.0714 

0.2575 

Ss  (in  Manual  Response  Group) 

X35 

PI  x03 

0.0714 

0.2575 

X09 

Subject  Ml 

0.0714 

0.2575 

X36 

P  1  x04 

0.0714 

0.2575 

XIO 

Subject  M2 

0.0714 

0.2575 

X37 

P  1  x05 

0.0714 

0.2575 

XU 

Subject  M3 

0.0714 

0.2575 

X38 

P  1  x06 

0.0714 

0.2575 

X12 

Subject  M4 

0.0714 

0.2575 

Criterion  Variables  (Y-set) 

X13 

Subject  M5 

0.0714 

0.2575 

Tactical  Assessment  Task  (TA) 

Order 

(of  Conditions) 

YOl 

TA  Percent  Correct. 

94.0978 

7.9708 

X14 

Order  1 

0.1429 

0.3499 

Y02 

TA  Md.  RT  Correct 

1.4772 

0.2117 

X15 

Order  2 

0.1429 

0.3499 

Y03 

TA  Percent  Incorrect. 

2.2875 

3.9264 

X16 

Order  3 

0.1429 

0.3499 

Y04 

TA  Md.  RT  Incorrect 

0.6970 

0.9531 

X17 

Order  4 

0.1429 

0.3499 

Y05 

TA  RMS  Error  Elev. 

17.6668 

9.4384 

X18 

Order  5 

0.1429 

0.3499 

Y06 

TA  RMS  Error  Azim. 

14.1387 

6.8633 

X19 

Order  6 

0.1429 

0.3499 

Y07 

TA  Stick  Manip.  Elev. 

0.4868 

0.1995 

Periods  (#1  vs  #3) 

Y08 

TA  Stick  Manip.  Azim. 

0.7382 

0.2414 

X20 

Period  1 

0.5000 

0.5000 

Communication  (Com)  Task 

Conditions 

Y09 

Com  Percent  Correct 

98.5624 

5.1916 

X21 

Condition  1 

0.1429 

0.3499 

YIO 

Com  Md.  RT  Correct 

2.0814 

0.6260 

X22 

Condition  2 

0.1429 

0.3499 

Yll 

Com  Md.  RT  Incorrect 

0.4430 

1.4628 

X23 

Condition  3 

0.1429 

0.3499 

Y12 

Com  RMS  Error  Elev. 

12.9298 

6.4181 

X24 

Condition  4 

0.1429 

0.3499 

Y13 

Com  RMS  Error  Azim. 

16.5623 

9.3000 

X25 

Condition  5 

0.1429 

0.3499 

Y14 

Com  Stick  Manip.  Elev. 

0.5008 

0.1830 

X26 

Condition  6 

0.1429 

0.3499 

Y15 

Com  Stick  Manip.  Azim. 

0.7295 

0.1993 
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Table  1  shows  the  means  and  standard  deviations  for  the  38  (X-set)  predictor  variables. 
Note  that  the  mean  of  any  predictor  variable  i  is  always  the  proportion  (p=i)  of  cases  that  received 
a  "1"  on  that  variable  while  the  standard  deviation  of  that  variable  is  the  square  root  of 
p=i(l-p=i).  For  example,  the  p=i  for  the  Verbal-Response  variable  is  .5714  (=  8/14  or  8  Ss  out  of 
14  Ss),  the  p=i  for  each  Subject  variable  is  .0714  (=  1/14  or  1  S  out  of  14  Ss);  the  p=i  value  for 
each  order  and  condition  is  .1429  (=  1/7),  etc.  Table  1  also  shows  the  means  and  standard 
deviations  of  the  15  (Y-set)  criteria  investigated.  TA  (and  Com)  RMS  and  Stick  Manipulation 
means  show  average  tracking  criterion  performance  prior  to  and  immediately  following  the  TA 
(or  Com)  tasks. 


Table  2.  [  R 

XX  ]: 

Original  Correlations  Among  Predictor  Variables 

Vb  Variable 

01 

02 

03 

04 

05 

06 

07 

08 

09 

10 

11 

12 

13 

14  15 

16 

17 

18  19 

01  Verb"Resp  Mode 

100 

24 

24 

24 

24 

24 

24 

24 

-32 

-32 

-32 

-32 

-32 

00  00 

00 

00 

00  00 

02  Subject  VI 

24 

100 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

00  00 

00 

00 

00  00 

03  Subject  V2 

24 

-08  100 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

00  00 

00 

00 

00  00 

04  Subject  V3 

24 

-08 

-08 

100 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

00  00 

00 

00 

00  00 

05  Subject  V4 

24 

-08 

-08 

-08  100 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

00  00 

00 

00 

00  00 

06  Subject  V5 

24 

-08 

-08 

-08 

-08  100 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

00  00 

00 

00 

00  00 

07  Subject  V6 

24 

-08 

-08 

-08 

-08 

-08  100 

-08 

-08 

-08 

-08 

-08 

-08 

00  00 

00 

00 

00  00 

08  Subject  V7 

24 

-08 

-08 

-08 

-08 

-08 

-08 

100 

-08 

-08 

-08 

-08 

-08 

00  00 

00 

00 

00  00 

09  Subject  Ml 

-32 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

100 

-08 

-08 

-08 

-08 

00  00 

00 

00 

00  00 

10  Subject  M2 

-32 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

-08  100 

-08 

-08 

-08 

00  00 

00 

00 

00  00 

1 1  Subject  M3 

-32 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

-08  100 

-08 

-08 

00  00 

00 

00 

00  00 

12  Subject  M4 

-32 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

-08  100 

-08 

00  00 

00 

00 

00  00 

13  Subject  M5 

-32 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

-08 

-08  100 

00  00 

00 

00 

00  00 

14  Order  1 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

100  -17 

-17 

-17 

-17  -17 

15  Order  2 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

-17  100 

-17 

-17 

-17  -17 

16  Order  3 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

-17  -17  100 

-17 

-17  -17 

17  Order  4 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

-17  -17 

-17  100 

-17  -17 

18  Order  5 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

-17  -17 

-17 

-17  100  -17 

19  Order  6 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

-17  -17 

-17 

-17 

-17  100 

20  Period  1 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00  00 

00 

00 

00  00 

21  Condition  1 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00  -08 

00 

00 

00  -08 

22  Condition  2 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

-08  -08 

25 

00 

08  00 

23  Condition  3 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

08  -08 

00 

33 

-08  -17 

24  Condition  4 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

17  08 

-08 

08 

-08  -08 

25  Condition  5 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

08  42 

-08 

-17 

00  -17 

26  Condition  6 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

-08  -08 

00 

-08 

25  17 

27  PI  Cl 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00  -06 

00 

00 

00  -06 

28  PI  C2 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

-06  -06 

17 

00 

06  00 

29  PI  C3 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

06  -06 

00 

23 

-06  -1 1 

30  PI  C4 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

11  06 

-06 

06 

-06  -06 

31  PI  C5 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

06  28 

-06 

-11 

00  -11 

32  PI  C6 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

-06  -06 

00 

-06 

17  11 

33  PI  01 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

68  -11 

-11 

-11 

-11  -11 

34  PI  01 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

-11  68 

-11 

-11 

-11  -11 

35  PI  01 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

-11  -11 

68 

-11 

-11  -11 

36  PI  01 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

-11  -11 

-11 

68 

-11  -11 

37  PI  05 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

-11  -11 

-11 

-11 

68  -11 

38  PI  06 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

-11  -11 

-11 

-11 

-11  68 

Note:  two  decimal  points  omitted. 
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The  intercorrelations  of  the  predictors  and  criteria  are  shown  in  Tables  2,  3,  and  4. 


Table  2  shows  the  intercorrelations  among  the  predictors  (i.e.,  [  Rxx  ])• 


Table  2.  (Cont.)  [  Rxx 

]:  Intercorrelations  of  Predictor  Variables 

Vb  Variable 

20 

21 

22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 

37  38 

01  Verb-Resp  Mode 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00  00 

02  Subject  VI 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00  00 

03  Subject  V2 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00  00 

04  Subject  V3 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00  00 

05  Subject  V4 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00  00 

06  Subject  V5 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00  00 

07  Subject  V6 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00  00 

08  Subject  V7 

00 

00 

00 

00 

00 

00 

TJO 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00  00 

09  Subject  Ml 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00  00 

10  Subject  M2 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00  00 

1 1  Subject  M3 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00  00 

12  Subject  M4 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00  00 

13  Subject  M5 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00  00 

14  Order  1 

00 

00 

-08 

08 

17 

08 

-08 

00 

-06 

06 

11 

06 

-06 

68 

-11 

-11 

-11 

-11  -11 

15  Order  2 

00 

-08 

-08 

-08 

08 

42 

-08 

-06 

-06 

-06 

06 

28 

-06 

-11 

68 

-11 

-11 

-11  -11 

16  Order  3 

00 

00 

25 

00 

-08 

-08 

00 

00 

17 

00 

-06 

-06 

00 

-11 

-11 

68 

-11 

-11  -11 

17  Order  4 

00 

00 

00 

33 

08 

-17 

-08 

00 

00 

23 

06 

-11 

-06 

-11 

-11 

-11 

68 

-11  -11 

18  Orders 

00 

00 

08 

-08 

-08 

00 

25 

00 

06 

-06 

-06 

00 

17 

-11 

-11 

-11 

-11 

68  -11 

19  Order  6 

00 

-08 

00 

-17 

-08 

-17 

17 

-06 

00 

-11 

-06 

-11 

11 

-11 

-11 

-11 

-11 

-11  68 

20  Period  1 

100 

00 

00 

00 

00 

00 

00 

28 

28 

28 

28 

28 

28 

28 

28 

28 

28 

28  28 

21  Condition  1 

00 

100 

-17 

-17 

-17 

-17 

-17 

68 

-11 

-11 

-11 

-11 

-11 

00 

-06 

00 

00 

00  -06 

22  Condition  2 

00 

-17  100 

-17 

-17 

-17 

-17 

-11 

68 

-11 

-11 

-11 

-11 

-06 

-06 

17 

00 

06  00 

23  Condition  3 

00 

-17 

-17  100 

-17 

-17 

-17 

-11 

-11 

68 

-11 

-11 

-11 

06 

-06 

00 

23 

-06  -11 

24  Condition  4 

00 

-17 

-17 

-17  100 

-17 

-17 

-11 

-11 

-11 

68 

-11 

-11 

11 

06 

-06 

06 

-06  -06 

25  Condition  5 

00 

-17 

-17 

-17 

-17  100 

-17 

-11 

-11 

-11 

-11 

68 

-11 

06 

28 

-06 

-11 

00  -11 

26  Condition  6 

00 

-17 

-17 

-17 

-17 

-17  100 

-11 

-11 

-11 

-11 

-11 

68 

-06 

-06 

00 

-06 

17  11 

27  PI  Cl 

28 

68 

-11 

-11 

-11 

-11 

-11 

100 

-08 

-08 

-08 

-08 

-08 

08 

00 

08 

08 

08  00 

28  PI  C2 

28 

-11 

68 

-11 

-11 

-11 

-11 

-08 

100 

-08 

-08 

-08 

-08 

00 

00 

31 

08 

15  08 

29  PI  C3 

28 

-11 

-11 

68 

-11 

-11 

-11 

-08 

-08  100 

-08 

-08 

-08 

15 

00 

08 

39 

00  -08 

30  PI  C4 

28 

-11 

-11 

-11 

68 

-11 

-11 

-08 

-08 

-08  100 

-08 

-08 

23 

15 

00 

15 

00  00 

31  PI  C5 

28 

-11 

-11 

-11 

-11 

68 

-11 

-08 

-08 

-08 

-08  100 

-08 

15 

46 

00 

-08 

08  -08 

32  PI  C6 

28 

-11 

-11 

-11 

-11 

-11 

68 

-08 

-08 

-08 

-08 

-08  100 

00 

00 

08 

00 

31  23 

33  PI  01 

28 

00 

-06 

06 

11 

06 

-06 

08 

00 

15 

23 

15 

00 

100 

-08 

-08 

-08 

-08  -08 

34  PI  02 

28 

-06 

-06 

-06 

06 

28 

-06 

00 

00 

00 

15 

46 

00 

-08  100 

-08 

-08 

-08  -08 

35  PI  03 

28 

00 

17 

00 

-06 

-06 

00 

08 

31 

08 

00 

00 

08 

-08 

-08  100 

-08 

-08  -08 

36  PI  04 

28 

00 

00 

23 

06 

-11 

-06 

08 

08 

39 

15 

-08 

00 

-08 

-08 

-08  100 

-08  -08 

37  PI  05 

28 

00 

06 

-06 

-06 

00 

17 

08 

15 

00 

00 

08 

31 

-08 

-08 

-08 

-08  100  -08 

38  PI  06 

28 

-06 

00 

-11 

-06 

-11 

11 

00 

08 

-08 

00 

-08 

23 

-08 

-08 

-08 

-08 

-08  100 

Note:  two  decimal  points  omitted. 


It  may  be  noted  in  Table  2  that  correlations  of  zero  exist  between  various  blocks  of  levels 
of  main  effects  and  various  blocks  of  levels  of  the  interaction  terms.  These  blocks  of  zero 
correlations  result  because  the  design  of  the  study  caused  these  main  effects  and  interactions  to 
be  independent  of  each  other.  For  example,  the  Verbal-Response  Mode  and  Subjects  are 
independent  of  all  other  effects.  Order,  Period,  and  Conditions  are  also  independent  of  each 
other. 
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Table  3  shows  the  correlations  of  the  38  predictors  with  the  15  criteria. 


Table  3.  [  Rxy  ]:  Original  Correlations  of  Predietors  With  Criteria 


data  predicated  on  TA  stimuli  data  predicated  on  Com  stimuli 

T A  criteria  Tracking  criteria  Com  criteria  Tracking  criteria 


Vb. 

Variable 

YI 

Y2 

Y3 

Y4 

Y5 

Y6 

Y7 

Y8 

Y9 

YIO 

Yll 

Y12 

Y13 

Y14 

Y15 

XOl 

Verb.-Resp.  Mode 

336 

-390 

-323 

-234 

010 

on 

105 

058 

-220 

650 

224 

018 

032 

089 

018 

X02 

Subject  VI 

040 

-363 

-009 

-031 

259 

463 

-098 

-268 

-203 

234 

175 

496 

216 

-113 

-263 

X03 

Subject  V2 

085 

-064 

-100 

-037 

-057 

-035 

189 

126 

-004 

136 

065 

-020 

-052 

313 

128 

X04 

Subject  V3 

010 

-093 

-085 

-037 

323 

129 

-120 

-067 

-092 

206 

061 

114 

350 

-140 

-021 

X05 

Subject  V4 

045 

198 

-039 

-073 

277 

206 

-129 

-205 

-073 

216 

139 

214 

294 

-144 

-204 

X06 

Subject  V5 

198 

-178 

-146 

-169 

-252 

-220 

-175 

-064 

-042 

188 

065 

-221 

-223 

-217 

-140 

X07 

Subject  V6 

107 

-079 

-055 

-020 

-162 

-163 

212 

255 

-162 

-318 

092 

-176 

-176 

189 

183 

X08 

Subject  V7 

023 

-095 

-070 

-071 

-150 

-183 

119 

287 

077 

334 

-084 

-224 

-138 

051 

239 

X09 

Subject  Ml 

046 

-040 

024 

098 

028 

038 

-049 

025 

077 

-247 

-084 

031 

004 

-015 

-022 

XIO 

Subject  M2 

-030 

-095 

084 

093 

-161 

-149 

043 

041 

077 

156 

-084 

-187 

-162 

-025 

024 

XII 

Subject  M3 

016 

292 

-146 

-169 

-043 

-005 

118 

-079 

077 

-373 

-084 

001 

-035 

194 

128 

X12 

Subject  M4 

-608 

122 

719 

295 

358 

196 

-425 

-409 

077 

-056 

-084 

230 

373 

-390 

-431 

XI3 

Subject  M5 

-200 

214 

055 

171 

039 

155 

no 

015 

039 

-360 

-010 

148 

013 

130 

-039 

X14 

Order  1 

-433 

-045 

226 

289 

236 

189 

-176 

-005 

-136 

052 

049 

205 

273 

-069 

001 

X15 

Order  2 

019 

047 

-001 

013 

123 

046 

-044 

-033 

026 

-004 

041 

037 

095 

-111 

-005 

X16 

Order  3 

073 

006 

-023 

006 

-042 

-042 

-046 

-069 

-059 

-010 

076 

-044 

-040 

050 

-112 

X17 

Order  4 

016 

021 

001 

-022 

025 

066 

086 

103 

-000 

-012 

-014 

064 

033 

097 

120 

X18 

Order  5 

077 

130 

-077 

-186 

-073 

-073 

025 

-069 

082 

-016 

-069 

-034 

-073 

001 

-070 

X19 

Order  6 

145 

-045 

-068 

-090 

-153 

-123 

096 

045 

113 

002 

-124 

-116 

-148 

-028 

051 

X20 

Period  1 

-035 

-129 

063 

073 

-038 

-138 

-028 

-099 

061 

034 

-059 

-180 

-070 

-048 

-108 

X21 

Condition  1 

058 

017 

-047 

-036 

-001 

039 

-093 

-138 

001 

033 

-034 

-046 

-059 

003 

-019 

X22 

Condition  2 

032 

002 

-031 

-109 

-097 

-115 

-059 

-126 

051 

-032 

-069 

-054 

-050 

-022 

-206 

X23 

Condition  3 

-097 

-118 

133 

101 

288 

379 

071 

157 

-029 

-on 

087 

308 

256 

154 

226 

X24 

Condition  4 

-093 

087 

047 

090 

-046 

-140 

006 

069 

062 

-018 

-069 

-075 

-002 

019 

049 

X25 

Condition  5 

-119 

015 

-013 

060 

128 

061 

000 

053 

-217 

031 

167 

038 

089 

-124 

-048 

X26 

Condition  6 

109 

064 

-066 

-110 

-158 

-165 

-024 

-030 

019 

-006 

041 

-075 

-104 

-024 

-005 

X27 

PI  xCl 

047 

-035 

-009 

-005 

-008 

-007 

-033 

-122 

039 

027 

-010 

-071 

-057 

014 

-008 

X28 

PI  xC2 

002 

-035 

-039 

-065 

-051 

-068 

051 

-033 

077 

-026 

-084 

-043 

-045 

on 

-088 

X29 

PI  xC3 

-043 

-137 

068 

-007 

033 

047 

-075 

-042 

-004 

-008 

065 

009 

016 

-111 

008 

X30 

PI  xC4 

-051 

020 

068 

065 

004 

-127 

015 

064 

077 

-021 

-084 

-091 

024 

012 

-028 

X31 

PI  xC5 

-148 

026 

-009 

106 

118 

069 

-086 

035 

-181 

048 

092 

009 

091 

-048 

-on 

X32 

PI  xC6 

062 

000 

007 

016 

-096 

-119 

-009 

-035 

034 

029 

-010 

-064 

-073 

044 

-013 

X33 

PI  xOl 

-420 

-043 

205 

242 

177 

064 

-093 

037 

-105 

065 

018 

055 

190 

-054 

-020 

X34 

PI  x02 

-006 

-012 

007 

029 

087 

035 

-108 

-045 

034 

-022 

-010 

-005 

074 

-076 

005 

X35 

PI  x03 

055 

-034 

006 

072 

-049 

-088 

-047 

-074 

000 

-006 

065 

-090 

-054 

054 

-095 

X36 

PI  x04 

062 

-010 

-009 

-061 

-054 

-047 

000 

-025 

077 

029 

-084 

-051 

-067 

-040 

-005 

X37 

PI  x05 

085 

068 

-085 

-111 

-051 

-063 

033 

-050 

077 

-000 

-084 

-046 

-068 

018 

-027 

X38 

PI  X  06 

085 

-096 

022 

-022 

-088 

-085 

094 

-008 

077 

001 

-084 

-088 

-084 

-009 

015 

Note:  three  decimal  points  omitted. 
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Table  4  shows  the  original  intercorrelations  among  the  15  criterion  variables. 


Table  4.  [  Ryy  ]:  Original  Correlations  Among  the  Criteria 

data  predicated  on  TA  stimuli 

data  predicated  on  Com  stimuli 

TA  criteria 

Tracking  criteria 

Com  criteria 

Tracking  criteria 

Vb. 

Variable 

Y1  Y2  Y3  Y4 

Y5 

Y6  Y7 

Y8 

Y9  YIO  Yll 

Y12 

Y13  Y14  Y15 

Y1 

TA  Percent  Correct 

1000  -136  -721  -489 

-529 

-408  352 

301 

-004  083  -on 

-431 

-548  266  289 

Y2 

”  Md.  RT  Correct 

-1361000  -012  -029 

-012 

-045  014 

038 

025  -370  -001 

-031 

on  057  071 

Y3 

"  Percent  Incorrect 

-721  -012  1000  631 

438 

300  -368 

-289 

073  -042  -072 

331 

455  -305  -301 

Y4 

"  Md.  RT  Incorrect 

-489  -029  631  1000 

243 

173  -150 

013 

030  -047  -018 

186 

236  017  -024 

Y5 

"  RMSE  in  Elev. 

-529  -012  438  243 

1000 

854  -404 

-421 

-177  114  201 

841 

968  -321  -357 

Y6 

”  RMSE  in  Azim. 

-408  -045  300  173 

854 

1000  -261 

-369 

-240  088  251 

950 

805  -153  -285 

Y7 

"  St.  Manip.  Elev. 

352  014  -368  -150 

-404 

-261  1000 

623 

-003  -no  -068 

-299 

-450  687  601 

Y8 

"  St.  Manip.  Azim. 

301  038  -289  013 

-421 

-369  623 

1000 

008  -127  -058 

-427 

-427  549  769 

Y9 

Com  Percent  Correct 

-004  025  073  030 

-177 

-240  -003 

008 

1000  -135  -826 

-244 

-172  051  097 

YIO 

"  Md.  RT  Correct 

083  -370  -042  -047 

114 

088  -no 

-127 

-135  1000  121 

088 

148  -139  -176 

YU 

"  Md.  RT  Incorrect 

-on  -001  -072  -018 

201 

251  -068 

-058 

-826  121  1000 

260 

195  -088  -116 

Y12 

”  RMS  Error  Elev. 

-431  -031  331  186 

841 

950  -299 

-427 

-244  088  260 

1000 

827  -184  -332 

Y13 

”  RMS  Error  Azim. 

-548  on  455  236 

968 

805  -450 

-427 

-172  148  195 

827 

1000  -346  -379 

Y14 

”  St.  Manip.  Elev. 

266  057  -305  017 

-321 

-153  687 

549 

051  -139  -088 

-184 

-3461000  596 

Y15 

"  St.  Manip.  Azim. 

289  071  -301  -024 

-357 

-285  601 

769 

097  -176  -116 

-332 

-379  596  1000 

Note:  three  decimal  points  omitted. 


3.2.1  Step  1 :  Obtaining  the  Criterion-Based  Diagonal  Factors 


Step  1  of  the  RDF  approach  was  accomplished.  This  consisted  of  obtaining  the 
15  Criterion-based  diagonal  factors.  These  unrotated  diagonal  factors  (D.c's)  explain  all  of  the 
criterion  variance,  all  of  the  correlations  among  the  criteria,  and  all  of  the  correlations  of  the 
criteria  with  the  predictor  variables. 


43 


NAWCADWAR-96-36-TR 


Table  5.  [Dye]:  Criterion  Loadings  on  Criterion-Based  Diagonal  Factors 


Vb. 

Variable  D1  D2 

D3 

D4 

D5 

D6 

D7 

D8 

D9  DIO 

Dll 

D12 

D13 

D14 

D15  h- 

YOl 

TA  Pet.  Correct  1000  000 

000 

000 

000 

000 

000 

000 

000 

000 

000 

000 

000 

000 

0001000 

Y02 

"  Md.RT  Correct  -136  991 

000 

000 

000 

000 

000 

000 

000 

000 

000 

000 

000 

000 

000  1000 

Y03 

"  Pet.  Incorrect  -721-111 

684 

000 

000 

000 

000 

000 

000 

000 

000 

000 

000 

000 

0001000 

Y04 

"  Md.RT  Incorrect  -489  -096 

390 

774 

000 

000 

000 

000 

000 

000 

000 

000 

000 

000 

0001000 

Y06 

"  RMSE  in  Elev.  -529  -085 

070 

-067 

839 

000 

000 

000 

000 

000 

000 

000 

000 

000 

0001000 

Y07 

"RMSEinAzim,  -408  -101 

-007 

-043 

748 

512 

000 

000 

000 

000 

000 

000 

000 

000 

0001000 

Y08 

"St.Manip.Elev.  352  062 

-156 

116 

-231 

128 

874 

000 

000 

000 

000 

000 

000 

000 

0001000 

Y09 

"  St.Manip.Azim.  301  079 

-092 

263 

-276 

-041 

468 

727 

000 

000 

000 

000 

000 

000 

0001000 

YIO 

Com  Pet.  Correct  -004  025 

107 

-014 

-221 

-144 

-020 

-050 

957 

000 

000 

000 

000 

000 

0001000 

Yll 

"  Md.RT  Correct  083  -362 

-032 

-037 

151 

-058 

-086 

-051 

-107 

901 

000 

000 

000 

000 

0001000 

Y12 

"  Md.RT  Incorrect  -01 1  -003 

-118 

029 

244 

126 

-052 

033 

-774 

003 

554 

000 

000 

000 

0001000 

Y13 

"RMSE  in  Elev.  -431  -091 

015 

-051 

716 

444 

-028 

-064 

-028 

-002 

on 

296 

000 

000 

0001000 

Y14 

"  RMSE  in  Azim.  -548  -064 

078 

-089 

788 

-034 

-050 

018 

-014 

049 

-003 

098 

211 

000 

0001000 

Y15 

"St.Manip.Elev.  266  094 

-151 

278 

-171 

202 

533 

119 

082 

-027 

-013 

042 

030 

665 

0001000 

Y16 

"  St.Manip.Azim.  289  112 

-116 

225 

-204 

Oil 

457 

458 

105 

-055 

009 

006 

-022 

126 

590  1000 

Note:  three  decimal  points  omitted. 


Table  5  shows  the  correlations  of  the  criterion  variables  with  the  15  unrotated  diagonal 
factors  (i.e.,  submatrix  [  Dye  ])•  It  can  be  seen  in  Table  5  that  the  communality  (i.e.,  the  sum  of 
the  squares  of  the  loadings  of  a  variable  across  all  factor),  h^,  of  each  criterion  variable  is  equal 
to  1.  This  shows  that  these  criterion-based  diagonal  factors  do,  in  fact,  explain  all  of  the  variance 
of  each  criterion. 


Table  6  shows  the  predictors'  loadings  on  (i.e.,  correlations  with)  the  unrotated  criterion- 
based  diagonal  factors  (i.e.,  submatrix  [  Dye]). 
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Table  6.  [Dxc]:  Predictor  Loadings  on  Criterion-Based  Diagonal  Factors 


Vb. 

Variable 

D1 

D2 

D3 

D4 

D5 

D6 

D7 

D8 

D9 

DIO 

Dll 

D12 

D13 

D14 

D15 

XOl 

Verbal-Resp.Mode 

336 

-348 

-175 

-046 

199 

-077 

048 

014 

-165 

490 

074 

069 

064 

096 

-007 

605 

X02 

Subject  VI 

040 

-360 

-029 

-045 

296 

428 

-085 

-141 

-077 

060 

-023 

215 

-158 

-079 

-087 

527 

X03 

Subject  V2 

085 

-053 

-066 

032 

-012 

008 

166 

013 

008 

140 

132 

082 

013 

277 

-032 

165 

X04 

Subject  V3 

010 

-093 

-129 

on 

393 

-335 

-006 

027 

-038 

095 

-072 

-066 

185 

-030 

104 

359 

X05 

Subject  V4 

045 

206 

024 

-053 

373 

-071 

-060 

-124 

-017 

235 

089 

on 

074 

-040 

-074 

287 

X06 

Subject  V5 

198 

-152 

-030 

-097 

-196 

-024 

-310 

002 

-092 

114 

052 

-050 

027 

-126 

-048 

254 

X07 

Subject  V6 

107 

-065 

023 

023 

-132 

-050 

177 

141 

-197 

-367 

-023 

-035 

027 

070 

-039 

270 

X08 

Subject  V7 

023 

-093 

-093 

-042 

-170 

-114 

094 

268 

052 

377 

-005 

-101 

-027 

-038 

106 

310 

X09 

Subject  Ml 

046 

-034 

077 

112 

061 

026 

-061 

052 

094 

-286 

-049 

004 

-028 

-041 

-088 

135 

XIO 

Subject  M2 

-030 

-100 

075 

051 

-223 

-003 

017 

-025 

022 

181 

-007 

-156 

-009 

-067 

027 

133 

Xll 

Subject  M3 

016 

297 

-148 

-096 

-006 

062 

082 

-184 

088 

-293 

-044 

008 

117 

167 

161 

327 

X12 

Subject  M4 

-608 

040 

417 

-208 

-003 

-099 

-128 

-112 

003 

-008 

-042 

-029 

-051 

-013 

-090 

641 

X13 

Subject  M5 

-200 

189 

-100 

169 

-039 

251 

106 

-059 

076 

-264 

032 

022 

-020 

-025 

-176 

302 

X14 

Order  1 

-433 

-105 

-143 

159 

023 

-018 

-057 

153 

-114 

041 

-139 

059 

097 

019 

104 

329 

X15 

Order  2 

019 

050 

027 

022 

164 

-122 

001 

-009 

042 

-014 

094 

-045 

-087 

-103 

043 

079 

X16 

Order  3 

073 

016 

046 

033 

-004 

-012 

-078 

-084 

-075 

-030 

044 

-041 

022 

132 

-124 

066 

X17 

Order  4 

016 

023 

022 

-026 

038 

089 

096 

103 

026 

on 

-017 

050 

048 

031 

054 

041 

X18 

Order  5 

077 

142 

-010 

-170 

-037 

-013 

001 

-097 

065 

032 

001 

108 

-081 

022 

-075 

096 

X19 

Order  6 

145 

-026 

050 

-053 

-102 

016 

040 

-033 

091 

008 

-033 

029 

015 

-134 

023 

069 

X20 

Period  1 

-035 

-135 

033 

038 

-081 

-201 

001 

-159 

007 

-017 

-002 

-231 

-062 

037 

-059 

157 

X21 

Condition  1 

058 

026 

-002 

-005 

038 

072 

-131 

-111 

012 

021 

-083 

-295 

-141 

114 

111 

180 

X22 

Condition  2 

032 

007 

-010 

-115 

-103 

-057 

-087 

-134 

on 

-042 

-046 

150 

109 

065 

-204 

138 

X23 

Condition  3 

-097 

-132 

072 

016 

264 

252 

174 

277 

082 

-040 

no 

-081 

025 

096 

162 

336 

X24 

Condition  4 

-093 

075 

-016 

076 

-098 

-183 

027 

030 

017 

033 

-024 

173 

064 

008 

007 

101 

X25 

Condition  5 

-119 

-001 

-145 

076 

096 

-111 

054 

071 

-200 

006 

-031 

-087 

-144 

-165 

-074 

172 

X26 

Condition  6 

109 

079 

031 

-079 

-120 

-051 

-085 

-055 

-026 

015 

no 

256 

073 

-000 

032 

138 

X27 

PI  X  Cl 

047 

-029 

032 

004 

015 

-002 

-045 

-147 

032 

004 

034 

-249 

-102 

096 

100 

122 

X28 

PI  X  C2 

002 

-035 

-061 

-056 

-062 

-053 

048 

-087 

061 

-034 

-027 

062 

004 

-006 

-138 

053 

X29 

PI  X  C3 

-043 

-144 

030 

-069 

-on 

039 

-052 

036 

-001 

-063 

114 

-125 

-017 

-065 

no 

083 

X30 

PI  X  C4 

-051 

013 

048 

029 

-028 

-242 

070 

035 

035 

-008 

-023 

073 

029 

028 

-124 

096 

X31 

PI  X  C5 

-148 

006 

-169 

129 

072 

-079 

-056 

099 

-161 

033 

-129 

-201 

-061 

-003 

-004 

179 

X32 

PI  X  C6 

062 

009 

077 

022 

-079 

-063 

-037 

-083 

-005 

034 

042 

139 

010 

085 

003 

058 

X33 

PI  X  01 

-420 

-101 

-158 

114 

-042 

-161 

039 

124 

-116 

063 

-128 

-061 

023 

019 

003 

307 

X34 

PI  X  02 

-006 

-013 

002 

032 

102 

-085 

-085 

019 

046 

-051 

012 

-142 

010 

-005 

079 

058 

X35 

PI  X  03 

055 

-027 

063 

093 

-024 

-090 

-068 

-118 

-031 

-036 

115 

-065 

002 

144 

-098 

094 

X36 

PI  X  04 

062 

-002 

053 

-066 

-035 

004 

-016 

-032 

064 

036 

-031 

-019 

-068 

-040 

021 

027 

X37 

PI  X  05 

085 

080 

-022 

-069 

-003 

-042 

009 

-099 

068 

022 

-038 

039 

-103 

022 

-028 

051 

X38 

PI  X  06 

085 

-085 

108 

-040 

-072 

-013 

086 

-092 

049 

-019 

-009 

-024 

051 

-070 

022 

060 

Note:  three  decimal  points  omitted. 


3.2.2  Step  2:  Rotating  the  Criterion-Based  Diagonal  Factors 


Step  2  of  the  RDF  approach  requires  the  rotation  of  the  criterion-based  diagonal  factors  to 
a  meaningful  structure.  The  Varimax  technique  was  attempted,  but  did  not  result  in  the  desired 
meaningful  set  of  rotated  factors  that  clearly  showed  between-tasks,  within-task,  and  within- 
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criterion  factors  discussed  previously.  Ultimately,  a  graphical  rotation  technique  from  "The  RDF 
Analysis  Program"  was  used.  It  did  result  in  the  desired  type  of  factor  structure.  The  results  of 
step  2  of  the  RDF  approach  are  shown  in  Table  4. 

Table  7  shows  the  15  criterion  variables'  loadings  on  (correlations  with)  the  rotated 
criterion-based  diagonal  factors.  It  represents  submatrix  [  Fyc  ]  discussed  in  subsection  2.7.2. 
Each  of  these  loadings  represent  the  relationship  between  a  specific  criterion  variable  and  one  of 
these  independent  factors. 


Table?.  [  Fyc  ]:  The  Criterion  Loadings  on  the  RDFs 


Rotated  Diagonal  Factors 


Vb. 

Variable 

1 

2 

3  4 

5  6 

7 

8 

9 

10 

11 

12 

13  14  15  h- 

YOl 

TA  Pet.  Correct 

552 

005 

070  -003 

536  -010 

-104 

623 

051 

025 

-002 

-002 

000  002  008  1000 

Y02 

"  Md.  RT  Correct 

001 

040 

044  012 

-028  -001 

369 

-064 

-924 

021 

023 

-005 

-004  -010  010  1000 

Y03 

"  Pet.  Incorrect 

-469 

002 

-no  -015 

-875  024 

-017 

018 

026 

017 

003 

022 

-001  -017  008  1000 

Y04 

"  Md.  RT  Incorrect 

-266 

-003 

136  047 

-611  -019 

-028 

-017 

030 

-728 

004 

-039 

-012  021  018  1000 

Y05 

”  RMSE  Elev. 

-997 

006 

024  -039 

031  -010 

006 

004 

013 

007 

-036 

019 

012  026  -005  1000 

Y06 

"  RMSE  Azim. 

-846 

-406 

056  098 

099  -066 

008 

005 

026 

007 

-302 

023 

-008  017  -005  1000 

Y07 

"  St.Manip.Elev. 

429 

-034 

587  199 

129  -029 

014 

010 

013 

030 

-018 

641 

016  017  0181000 

Y08 

"  St.Manip.Azim. 

454 

005 

835  -303 

-015  -055 

-003 

-002 

-006 

-034 

006 

000 

003  -003  -009  1000 

Y09 

Com  Pet.  Correct 

162 

039 

-017  002 

-143  974 

031 

-003 

-009 

-009 

026 

-024 

018  -004  0111000 

YIO 

"  Md.RT  Correct 

-117 

025 

-104  -025 

141  -070 

-973 

-037 

006 

007 

on 

001 

-006  -012  -001  1000 

Yll 

”  Md.RT  Incorrect 

-195 

-026 

-014  -010 

167  -779 

-016 

-000 

-010 

-017 

-021 

-016 

-569  005  -005  1000 

Y12 

”  RMSE  Elev. 

-848 

-503 

-006  123 

073  -078 

012 

002 

015 

003 

-012 

025 

-018  -013  -004  1000 

Y13 

”  RMSE  Azim. 

-976 

-009 

-003  -055 

006  -015 

-028 

-023 

-020 

020 

048 

-024 

013  -199  -014  1000 

Y14 

”  St.Manip.  Elev. 

317 

-017 

712  620 

079  021 

022 

007 

-015 

-035 

-006 

001 

013  -001  -0091000 

Y15 

”  St.Manip.  Azim. 

376 

004 

709  -048 

059  050 

065 

006 

-014 

-017 

004 

026 

001  007  5851000 

Factor  Type 

BT 

WT 

WT  WT 

WT  WT 

BT 

WC 

WC 

WC 

WC 

WC 

WC  WC  WC 

Note:  three  decimal  points  omitted. 


Table  7  does  show  the  type  of  structure  desired.  Factor  1,  for  example,  is  a  between-tasks 
factor  since  it  has  high  loadings  on  both  the  tactical  assessment  task  (TA),  the  tracking  task  while 
accomplishing  a  TA  task,  and  the  tracking  task  while  accomplishing  a  Com  task.  It  even  shows 
some  non-zero  loadings  on  the  Com  task.  Factors  2,  3,  and  4  are  all  within-task  factors  since 
they  deal  with  multiple  criteria  for  the  tracking  task  (for  both  when  the  Ss  performed  the  TA  task 
and  when  they  performed  the  Com  task).  Factor  5  is  also  a  within-task  factor  since  it  has  high 
loadings  on  more  than  one  criterion  for  the  TA  task,  but  no  other  high  loadings  for  any  other 
tasks.  Factor  6  is  also  a  within-task  factor  since  it  has  high  loadings  on  more  than  one  criterion 
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for  the  Com  task,  but  no  high  loadings  on  criteria  for  any  other  task.  Factor  7  is  a  between-tasks 
factor  since  it  has  high  loadings  on  criteria  for  both  TA  and  Com  tasks.  Factors  8  through  15  are 
all  within-criterion  factors  since  only  one  criterion  from  each  task  has  high  loadings  on  any  of 
these  factors.  Table  8  shows  the  predictor  loadings  on  the  rotated  diagonal  factors. 

Table  8.  [  Fxc  ]:  The  Predictor  Loadings  on  the  RDFs 

Rotated  Diagonal  Factors 

Vb.  Variable  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  h' 

XOl  Verb-Resp  Mode  -002  048  082  039  377  -149  -610  087  172  055  090  010  -073  -063  -016  605 

X02  Subject  VI  -254  -487  -148  095  170  -112  -180  109  276  -009  -051  053  009  142  -082  527 

X03  Subject  V2  050  019  213  241  062  005  -162  -014  022  046  089  -010  -130  -009  -038  165 

X04  Subject  V3  -314  308  040  -135  271  -015  -125  029  057  -060  084  -005  088  -160  093  359 

X05  Subject  V4  -279  069  -096  -007  207  003  -152  153  -293  -004  033  052  -086  -066  -068  287 

X06  Subject  V5  253  -004  -274  -160  071  -073  -174  022  105  038  -045  -157  -063  -041  -053  254 

X07  Subject  V6  173  095  200  -000  -058  -211  294  072  215  047  020  027  026  -015  -046  270 

X08  Subject  V7  161  101  192  -195  -020  060  -376  -176  -021  098  -043  -024  008  017  099  310 

X09  Subject  Ml  -022  -047  023  -057  -019  074  240  144  124  -117  -024  -065  049  039  -085  135 

XIO  Subject  M2  161  063  -053  -035  -171  034  -198  -072  030  -049  -126  061  004  011  033  133 

XI 1  Subject  M3  035  020  -004  269  140  077  390  -050  -152  094  -003  045  051  -110  156  327 

X12  Subject  M4  -383  092  -292  -019  -583  046  054  -090  -095  162  036  -013  044  038  -086  641 

X13  SubJectM5  -039  -203  078  117  -069  039  349  -158  -092  -163  -101  102  -039  034  -169  302 

X14  Order  1  -244  -028  086  -066  -149  -128  -030  -367  068  -150  042  -136  134  -101  087  329 

XI 5  Order 2  -116  098  -015  -114  068  053  026  086  -047  -047  019  052  -086  095  053  079 

X16  Order  3  036  071  -061  131  008  -069  014  090  -007  -031  -011  -077  -049  -008  -125  066 

X17  Order4  -023  -090  142  005  -004  009  -004  032  -021  065  -012  012  017  -057  050  041 

X18  Order5  068  -040  -087  060  063  081  018  029  -138  156  111  041  002  066  -070  096 

XI 9  Order  6  157  -058  -049  -074  011  096  -023  085  029  035  006  108  035  -027  029  069 

X20  Period  1  040  295  -121  059  -076  037  -035  -028  132  -066  -069  054  009  095  -052  157 

X21  Condition  1  007  111  -118  129  056  004  -022  048  -039  -006  -268  -103  078  167  115  180 

X22  Condition  2  083  -011  -158  101  011  033  040  -006  012  079  160  -028  045  -119  -210  138 

X23  Condition  3  -276  -151  348  -000  -041  030  -008  069  130  071  -222  -035  -110  -017  154  336 

X24  Condition  4  033  058  047  -033  -080  036  004  -105  -073  -065  240  -011  028  -079  004  101 

X25  Condition  5  -110  101  038  -190  060  -197  010  -148  -006  -093  -010  086  033  160  -070  172 

X26  Condition  6  142  -084  -106  014  015  -006  003  054  -070  067  245  -038  -111  -108  030  138 

X27  PlxCl  011  155  -099  133  015  038  -025  056  022  -014  -187  -013  -033  128  108  122 

X28  PlxC2  049  011  -046  041  027  074  028  -056  052  035  084  079  031  -003  -137  053 

X29  PlxC3  -027  012  -060  -080  -049  -002  014  -011  152  056  -133  -018  -114  020  108  083 

X30  PlxC4  -009  162  065  -041  -078  059  008  -009  -003  -017  189  020  032  -027  -125  096 

X31  PlxC5  -105  159  051  -087  046  -169  -014  -191  -022  -130  -126  -091  125  089  -010  179 

X32  PlxC6  082  001  -053  097  -044  011  -044  067  -018  -016  163  -033  -043  -025  005  058 

X33  PlxOl  -178  155  096  -078  -161-111  -050  -401  067  -102  031  -046  130  -010  -009  307 

X34  Plx02  -083  133  -026  -056  041  048  040  037  026  -050  -087  -075  -008  007  078  058 

X35  Plx03  040  148  -065  145  -025  -016  003  083  036  -093  014  -065  -116  017  -095  094 

X36  Plx04  056  -011  -061  -022  -008  070  -039  057  -012  056  -019  024  032  062  026  027 

X37  Plx05  050  015  -058  058  078  079  002  038  -077  056  067  040  042  100  -022  051 

X38  Plx06  088  022  -050  006  -054  060  -020  099  090  027  -014  142  014  -052  028  060 

_ Factor  Type  BT  WT  WT  WT  WT  WT  BT  WC  WC  WC  WC  WC  WC  WC  WC 

Note:  three  decimal  points  omitted;  bold  indicates  five  percent  of  the  variance  is  explained. 
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3.2.3  Step  3:  Predicting  the  RDFs  Using  Predictor  Variables 

Step  3  in  the  RDF  approach  is  to  find  the  P-weights  for  the  predictor  variables  to  predict 
the  rotated  diagonal  factors.  Subsection  2.7.3  had  indicated  that  this  can  be  accomplished  hy 
performing  a  predictor-based  Diagonal  Factor  Analysis  (DFA)  on  the  following  matrix: 


augmented  combined  matrix 

diagonal  factors 

the  residual  augmented  matrix 

[Rxx][-Fxc][  Imm] 

[Dxp] 

[0mm]  [Omn  ][0mm] 

[-Fxc]  [Rcc]  [  Onm]  yields 

[Dcp] 

and 

[0nm][Ucc  ][PCX] 

[I  mm]  [O'nm]  [0mm] 

[Dip] 

[0mm][P’cx][R''xx] 

Submatrix  [Rxx  ]  contains  the  intercorrelations  among  the  X-set  predictor  variables 
which  were  previously  computed  and  shown  in  Table  2.  Submatrices  [-Fxc  ]  and  [-F'xc  ] 
contain  the  reflected  correlations  of  the  predictor  variables  with  the  rotated  diagonal  factors 
(RDFs),  and  were  shown  in  Table  8.  Submatrix  [Rcc  ]  contains  the  correlations  among  the 
15  independent  RDFs.  Because  these  independent  factors  are  unrelated  to  each  other,  this 
submatrix  can  be  represented  by  an  n’^n  identity  submatrix  (i.e.,  "1"  in  the  main  diagonal  and  "0" 
in  all  other  cells).  The  identity  submatrix  (i.e.,  [Imm  ]),  zero  submatrix  (i.e.,  [Onm  ]),  and  their 
transposes  (i.e.,  [I'mm  ]  and  [O’nm  ]),  and  [0  mm  ]  were  needed  to  augment  the  combined  matrix. 
These  submatrices  were  created  and  augmented  to  the  combined  matrix.  The  MMC  technique 
was  then  accomplished  by  sequentially  removing  the  predictor-based  diagonal  factors  and 
removing  their  effects  from  the  augmented  combined  matrix.  When  all  of  the  predictor-based 
diagonal  factors  have  been  removed  from  the  combined  augmented  matrix,  it  appears  as  shown 
above  as  the  residual  augmented  matrix. 

Extraction  of  the  38  predictor-based  diagonal  factors  resulted  in  the  submatrices 
described  earlier  including  five  zero  matrices  (i.e.,  [Omm],  [Omn  ],  [0mm  ],  [Onm],  and  [0mm]  )•  These 
submatrices  show  that  all  of  the  variance  related  to  the  predictor  variables  was  removed  from  the 
combined  augmented  matrix;  the  predictor-based  diagonal  factors  accounted  for  that  variance. 
However,  the  residual  matrix  also  contains  four  important  submatrices:  [  Ucc  ]  (indicating  the 
unexplained  variance  of  the  rotated  diagonal  factors),  [  Pcx  ]  (the  standard-score  beta  weights). 
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[  P'cx  ]  the  transpose  of  [  Pcx  ],  and  [  R''xx  ]  (the  inverse  of  matrix  [  Rxx  ]  )•  Together,  as 
explained  earlier,  these  matrices  contain  information  needed  to  create  the  source  table,  compute 
the  B-weights,  and  conduct  the  significance  tests. 

Table  9  shows  the  submatrix  [  D'cp  ].  It  is  the  transpose  of  [  Dcp  ]  and  contains  the 
relationships  between  the  predictor-based  diagonal  factors  (PDFs)  and  the  rotated  diagonal 
factors  (RDFs).  Both  the  PDFs  and  the  RDFs  are  sets  of  independent  factors.  Thus,  the 
communalities  for  the  PDFs  are  the  eigenvalues  of  the  RDFs  and  the  RDF  communalities  are  the 
eigenvalues  of  the  PDFs. 
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Table  9.  [  D'cp  ]:  The  PDF  Correlations  on  the  RDFs 


Rotated  Diagonal  Factors 

PDFs*  FI  F2  F3  F4  F5  F6  F7  F8  F9  FIO  Fll  F12  F13  F14  F15  h’ 

01  Verb-Resp  Mode  002  -048  -082  -039  -377  149  610  -087  -172  -055  -090  -010  073  063  016  605 

02  Subject  VI  261  514  173  -088  -082  078  034  -091  -242  023  075  -052  -027  -162  081  493 

03  Subject  V2  -015  066  -176  -254  018  -031  021  023  -015  -031  -059  005  113  -030  047  123 

04  Subject  V3  367  -227  -026  097  -199  -016  -014  -018  -054  075  -065  001  -096  124  -082  281 

05  Subject  V4  411-020  115  -020  -175  -039  012  -156  310  032  -025  -060  070  051  072  354 

06  Subject  V5  -061  055  346  145  -075  035  040  -055  -042  -005  055  155  064  038  077  199 

07  Subject  V6  008  -038  -070  016  046  212  -496  -137  -189  -018  001  002  -014  023  100  366 

08  Subject  V7  031  -075  -110  294  028  -002  054  101  -006  -101  088  073  -000  -004  -024  143 

09  SubjectMl  024  033  -052  047  -107  -028  -047  -181  -189  105  -005  065  -027  -020  095  115 

10  SubjectM2  -168  -078  018  034  032  009  414  011  -130  055  104  -056  015  006  -011  243 

11  Subject  M3  -077  -052  -031  -295  -304  -036  -120  -011  034  -088  -003  -054  -032  142  -151  261 

12  Subject  M4  385  -159  296  -077  423  -014  227  032  -019  -204  -050  -008  -038  026  073  551 

13  SubjectM5  198  127  008  -318  032  -015  -008  148  -036  089  100  -171  048  051  239  290 

14  Order  1  244  028  -086  066  149  128  030  367  -068  150  -042  136  -134  101  -087  329 

15  Order  2  159  -095  000  127  -044  -032  -021-025  036  073  -026  -030  065  -079  -068  078 

16  Order3  037  -088  048  -098  008  087  -014  -035  003  073  -001  097  041  010  100  057 

17  Order4  111  059  -154  008  024  029  002  017  019  -009  -000  030  -018  065  -057  051 

18  Order  5  054  026  041  -051  -042  -041  -023  027  159  -115  -140  010  -008  -048  055  076 

19  Order  6  -023  069  022  086  -005  -090  014  -026  057  -041  -105  -071  -058  039  -035  049 

20  Period  1  -040  -295  121-059  076  -037  035  028  -132  066  069  -054  -009  -095  052  157 

21  Condition  1  008  -109  122  -109  -062  -021  022  -055  058  003  249  093  -083  -170  -127  173 

22  Condition  2  -065  -005  165  -086  -009  -036  -034  035  -035  -060  -105  037  -068  098  162  100 

23  Condition  3  240  117  -260  -034  006  -077  002  -127  -144  -097  276  035  113  -029  -139  302 

24  Condition  4  -043  -049  -052  -045  054  -100  -010  -000  039  -006  -136  026  003  049  -008  045 

25  Condition  5  060  -088  -102  118  -049  184  -008  131  -036  032  075  -071  -070  -159  094  142 

26  Condition  6  -065  023  011-016  -017  112  -020  016  -014  -057  -209  108  125  080  -023  100 

27  PI  X  Cl  007  003  -022  -043  003  -037  000  -046  -017  -012  -021-062  130  017  -065  032 

28  PlxC2  029  095  -145  058  -060  -068  -016  058  -014  -002  005  -134  025  -074  -041  072 

29  PlxC3  -221  -030  367  151  -012  030  -048  080  -047  -042  -059  -024  091  -020  -047  236 

30  PlxC4  017  -051-045  099  -011  -051-039  -085  -034  -085  -089  -065  036  -018  146  070 

31  PlxC5  023  -019  -050  038  -064  044  -012  134  060  037  117  203  -099  052  -069  106 

32  PlxC6  009  030  -074  -021  004  -021  028  -000  012  053  -021  077  -037  -042  -018  021 

33  PlxOl  047  -120  -127  047  067  048  040  213  022  -018  -030  -084  -048  -057  067  107 

34  Plx02  000  007  -004  009  014  -014  -047  037  -068  -032  068  066  -019  096  -065  033 

35  Plx03  006  -074  -016  -073  043  -010  -004  -023  -010  074  -050  082  111  041  007  042 

36  Plx04  -004  022  039  -016  013  -088  072  -026  076  -008  027  046  -061-094  -020  038 

37  Plx05  000  007  -012  003  -067  -053  010  -021  045  040  -015  048  -057  -045  -069  024 

38  PI  X  06  024  -000  -000  -028  032  -018  -009  -043  -022  -014  -001  025  -005  093  -114  028 

=  Communality  827  567  668  517  575  206  884  393  383  185  346  239  176  227  297  6.490 

Factor  Type _ BT  WT  WT  WT  WT  WT  BT  WC  WC  WC  WC  WC  WC  WC  WC 

"^PDF factors  represent  the  predictors  named  with  all  of  the  previous  predictors'  variance  removed. 

Note:  three  decimal  points  omitted. 
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Table  10  shows  the  squares  of  these  values.  They  represent  the  proportion  of  variance  of 
the  RDFs  that  can  be  explained  by  each  PDFs,  or  conversely,  the  proportion  of  variance  of  the 
PDFs  that  can  be  explained  by  each  RDF.  They  also  represent  the  Multiple  R^s  of  the  RDFs. 


Table  10. 

The  PDF  Squared  Correlations  on  the  RDFs 

PDFs* 

FI 

F2 

F3 

F4 

Rotated  Diagonal  Factors 
F5  F6  F7  F8  F9  FIO 

Fll 

F12 

F13 

F14 

F15 

h' 

01 

Verb-Resp  Mode 

000 

002 

007 

002 

142 

022 

372 

008 

030 

003 

008 

000 

005 

004 

000 

605 

02 

Subject  VI 

068 

264 

030 

008 

007 

006 

001 

008 

058 

000 

006 

003 

000 

026 

006 

493 

03 

Subject  V2 

000 

004 

031 

064 

000 

000 

000 

000 

000 

000 

004 

000 

013 

000 

002 

123 

04 

Subject  V3 

134 

051 

000 

009 

040 

000 

000 

000 

003 

006 

004 

000 

009 

015 

007 

281 

05 

Subject  V4 

169 

000 

013 

000 

031 

001 

000 

024 

096 

001 

000 

004 

005 

003 

005 

354 

06 

Subject  V5 

004 

003 

120 

021 

006 

001 

002 

003 

002 

000 

003 

024 

004 

001 

006 

199 

07 

Subject  V6 

000 

001 

005 

000 

002 

045 

246 

019 

036 

000 

000 

000 

000 

000 

010 

366 

08 

Subject  V7 

000 

006 

012 

086 

000 

000 

003 

010 

000 

010 

008 

005 

000 

000 

000 

143 

09 

Subject  Ml 

000 

001 

003 

002 

012 

000 

002 

033 

036 

on 

000 

004 

000 

000 

009 

115 

10 

Subject  M2 

028 

006 

000 

001 

001 

000 

172 

000 

017 

003 

oil 

003 

000 

000 

000 

243 

11 

Subject  M3 

006 

003 

000 

087 

093 

001 

014 

000 

001 

008 

000 

003 

001 

020 

023 

261 

12 

Subject  M4 

148 

025 

088 

006 

179 

000 

052 

001 

000 

042 

003 

000 

001 

000 

005 

551 

13 

Subject  M5 

039 

016 

000 

101 

001 

000 

000 

022 

001 

008 

010 

029 

002 

003 

057 

290 

14 

Order  1 

060 

000 

007 

004 

022 

016 

000 

135 

005 

022 

002 

018 

018 

010 

008 

329 

15 

Order  2 

025 

009 

000 

016 

002 

001 

000 

000 

001 

005 

000 

000 

004 

006 

005 

078 

16 

Order  3 

001 

008 

002 

010 

000 

008 

000 

001 

000 

005 

000 

009 

002 

000 

010 

057 

17 

Order  4 

012 

003 

024 

000 

000 

000 

000 

000 

000 

000 

000 

000 

000 

004 

003 

051 

18 

Order  5 

003 

000 

002 

003 

002 

002 

000 

000 

025 

013 

020 

000 

000 

002 

003 

076 

19 

Order  6 

000 

005 

000 

007 

000 

008 

000 

000 

003 

002 

on 

005 

003 

001 

001 

049 

20 

Period  1 

002 

087 

015 

003 

006 

001 

001 

000 

017 

004 

005 

003 

000 

009 

003 

157 

21 

Condition  1 

000 

012 

015 

012 

004 

000 

000 

003 

003 

000 

062 

009 

007 

029 

016 

173 

22 

Condition  2 

004 

000 

027 

007 

000 

001 

001 

001 

001 

004 

on 

001 

005 

010 

026 

100 

23 

Condition  3 

057 

014 

067 

001 

000 

006 

000 

016 

021 

009 

076 

001 

013 

000 

019 

302 

24 

Condition  4 

002 

002 

003 

002 

003 

010 

000 

000 

002 

000 

018 

000 

000 

002 

000 

045 

25 

Condition  5 

004 

008 

010 

014 

002 

034 

000 

017 

001 

001 

006 

005 

005 

025 

009 

142 

26 

Condition  6 

004 

000 

000 

000 

000 

013 

000 

000 

000 

003 

044 

012 

016 

006 

000 

100 

27 

PI  xCl 

000 

000 

000 

002 

000 

001 

000 

002 

000 

000 

000 

004 

017 

000 

004 

032 

28 

PI  xC2 

000 

009 

021 

003 

004 

005 

000 

003 

000 

000 

000 

018 

000 

005 

002 

072 

29 

PI  xC3 

049 

000 

135 

023 

000 

000 

002 

006 

002 

002 

003 

000 

008 

000 

002 

236 

30 

PI  xC4 

000 

003 

002 

010 

000 

003 

002 

007 

001 

007 

008 

004 

001 

000 

021 

070 

31 

PI  xC5 

000 

000 

002 

001 

004 

002 

000 

018 

004 

001 

014 

041 

010 

003 

005 

106 

32 

PI  xC6 

000 

000 

006 

000 

000 

000 

000 

000 

000 

003 

000 

006 

001 

002 

000 

021 

33 

PI  xOl 

002 

014 

016 

002 

004 

002 

002 

045 

000 

000 

000 

007 

002 

003 

004 

107 

34 

PI  x02 

000 

000 

000 

000 

000 

000 

002 

001 

005 

001 

005 

004 

000 

009 

004 

033 

35 

PI  x03 

000 

005 

000 

005 

002 

000 

000 

000 

000 

005 

002 

007 

012 

002 

000 

042 

36 

PI  x04 

000 

000 

001 

000 

000 

008 

005 

000 

006 

000 

000 

002 

004 

009 

000 

038 

37 

PI  x05 

000 

000 

000 

000 

004 

003 

000 

000 

002 

002 

000 

002 

003 

002 

005 

024 

38 

PI  x06 

000 

000 

000 

000 

001 

000 

000 

002 

000 

000 

000 

000 

000 

009 

013 

028 

Sums  =  Mul. 

827 

567 

668 

517 

575 

206 

884 

393 

383 

185 

346 

239 

176 

227 

297 

6.490 

L 

Factor  Type 

BT 

WT 

WT 

WT 

WT 

WT 

BT 

WC 

WC 

WC 

WC 

WC 

WC 

WC 

WC 

PDF  factors  represent  the  predictors  named  with  all  of  the  previous  predictors^  variance  removed. 
Note:  three  decimal  points  omitted. 
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Next,  using  the  values  in  Table  10,  the  sums  of  the  values  within  a  block  of  predictor 
variable  effects  are  now  added  together  to  obtain  the  "sums  of  squares"  (SSs)  for  those  effects. 
These  SSs  show  the  proportion  of  variance  explained  by  the  various  main  effects  and  interaction 
terms.  The  sum  of  the  SSs  for  a  given  criterion-based  RDF  is  the  total  explained  variance 
(i.e.,  R^)  for  that  factor,  and  1  -  is  the  residual  (i.e.,  error)  variance  for  that  factor.  If  the  SSs 
are  now  divided  by  the  number  of  levels  within  each  effect  (i.e.,  their  degrees  of  freedom),  then 
the  "mean  squares"  (MSs)  for  each  effect  can  be  obtained.  Finally,  if  the  MSs  are  divided  by  the 
residual  MS,  then  F-values  for  each  effect  are  obtained.  The  SSs,  MSs,  F-values,  and  their 
significance  levels  are  shown  in  Table  11. 


Table  11.  Source  Table  for  the  RDFs 


Effect 

dfs 

FI 

BT 

F2 

WT 

F3 

WT 

F4 

WT 

F5 

WT 

F6 

WT 

F7 

BT 

F8 

WC 

F9 

WC 

FIO 

WC 

Fll 

WC 

F12 

WC 

F13 

WC 

F14 

WC 

F15 

WC 

SS  Table 
Verb-Resp 

1 

000 

002 

007 

002 

142 

022 

372 

008 

030 

003 

008 

000 

005 

004 

000 

Subjects 

12 

599 

381 

303 

387 

371 

058 

492 

121 

251 

090 

048 

075 

038 

071 

132 

Orders 

6 

102 

026 

036 

040 

027 

036 

002 

138 

035 

048 

033 

035 

027 

025 

030 

Periods 

1 

002 

087 

015 

003 

006 

001 

001 

000 

017 

004 

005 

003 

000 

009 

003 

Conditions 

6 

071 

036 

123 

037 

010 

064 

002 

038 

028 

017 

217 

029 

045 

074 

071 

PxC 

6 

050 

014 

166 

040 

008 

012 

005 

037 

008 

013 

026 

074 

038 

on 

035 

PxO 

6 

003 

020 

018 

009 

012 

013 

009 

050 

014 

009 

009 

023 

022 

034 

027 

Residual 

157 

173 

433 

332 

483 

425 

794 

116 

607 

617 

815 

654 

761 

824 

773 

703 

Mul.  R2 

38 

827 

567 

668 

517 

575 

206 

884 

393 

383 

185 

346 

239 

176 

227 

297 

Total  Var. 

195 

1000 

1000 

1000 

1000 

1000 

1000 

1000 

1000 

1000 

1000 

1000 

1000 

1000 

1000 

1000 

MS  Table 

FI 

F2 

F3 

F4 

F5 

F6 

F7 

F8 

F9 

FIO 

Fll 

F12 

F13 

F14 

F15 

Verb-Resp 

1 

000 

002 

007 

002 

142 

022 

372 

008 

030 

003 

008 

000 

005 

004 

000 

Subjects 

12 

050 

032 

025 

032 

031 

005 

041 

010 

021 

008 

004 

006 

003 

006 

on 

Orders 

6 

017 

004 

006 

007 

004 

006 

000 

023 

006 

008 

006 

006 

005 

004 

005 

Periods 

1 

002 

087 

015 

003 

006 

001 

001 

000 

017 

004 

005 

003 

000 

009 

003 

Conditions 

6 

012 

006 

020 

006 

002 

on 

000 

006 

005 

003 

036 

005 

007 

012 

012 

PxC 

6 

008 

002 

028 

007 

001 

002 

000 

006 

001 

002 

004 

012 

006 

002 

006 

PxO 

6 

000 

003 

003 

001 

002 

002 

002 

008 

002 

001 

001 

004 

004 

006 

004 

Residual 

157 

001 

003 

002 

003 

003 

005 

000 

004 

004 

005 

004 

005 

005 

005 

004 

F  Table 

FI 

F2 

F3 

F4 

F5 

F6 

F7 

F8 

F9 

FIO 

Fll 

F12 

F13 

F14 

F15 

Verb-Resp 

1 

.000 

.84 

3.19 

.50 

52.9 

4.45 

503.0 

1.97 

7.60 

.580 

1.95 

.021 

1.02 

.807 

.061 

Subjects 

12 

45.8 

11.6 

12.0 

10.6 

11.5 

.952 

55.4 

2.63 

5.35 

1.46 

.967 

1.30 

.601 

1.21 

2.46 

Orders 

6 

15.6 

1.59 

2.83 

2.19 

1.65 

1.18 

.517 

6.00 

1.50 

1.56 

1.33 

1.20 

.881 

.834 

1.12 

Periods 

1 

1.44 

31.7 

7.01 

1.12 

2.14 

.269 

1.69 

.200 

4.44 

.847 

1.15 

.613 

.013 

1.84 

.618 

Conditions 

6 

10.9 

2.21 

9.71 

1.99 

.607 

2.13 

.521 

1.62 

1.21 

.565 

8.72 

.994 

1.44 

2.51 

2.67 

PxC 

6 

7.66 

.838 

13.2 

2.17 

.487 

.396 

1.12 

1.61 

.319 

.430 

1.05 

2.58 

1.24 

.371 

1.30 

PxO 

6 

.417 

1.22 

1.45 

.460 

.752 

.460 

2.04 

2.17 

.595 

.273 

.366 

.798 

.693 

1.15 

1.02 

Note:  three  decimal  points  omitted  in  SSs  Table  and  MSs  tabled  portions. 


3.2.4  Step  4:  Obtaining  the  B-Weights  for  the  Predictor  Variables 


To  convert  z-score  beta  weights  (i.e.,  P-weights)  to  raw  score  weights  (i.e.,  B-weights), 
and  to  compute  the  constants  to  be  added,  the  means  and  standard  deviations  of  the  RDFs'  scores 
must  be  known.  For  this  purpose,  the  means  of  RDFs  were  assumed  to  be  zero  and  their  standard 
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deviations  were  assumed  to  be  1.  Since  the  dichotomous  scores  of  all  38  predictors  are  either  "1" 
(if  applicable)  or  "0"  (if  not  applicable),  the  resulting  factor  score  B-weights  will  show  the  impact 
of  predictor  variables  in  terms  of  standard  deviation  units  of  the  RDFs.  Table  12  shows  the 
p-weights  to  apply  to  the  predictors'  z-scores  for  predicting  the  RDFs'  z-scores. 

Table  12.  [  P'cx  ]•  The  z-Score  P-Weights  for  Predicting  the  RDFs 


Rotated  Diagonal  Factors  (RDFs) 


Vb 

Variable 

FI 

F2 

F3 

F4 

F5 

F6 

F7 

F8 

F9 

FIO 

Fll 

F12 

F13 

F14 

F15 

01 

Verb-Resp 

-082 

015 

-107 

1.045 

078 

072-] 

1.132 

-155 

205 

-031 

073 

415 

-042 

-175 

-210 

02 

Subject  VI 

-427 

-444 

-166 

-131 

138 

-157 

109 

127 

285 

038 

-127 

-028 

001 

122 

-136 

03 

Subject  V2 

-145 

026 

169 

005 

038 

-048 

125 

013 

049 

089 

003 

-087 

-128 

-019 

-095 

04 

Subject  V3 

-483 

294 

009 

-344 

232 

-067 

160 

053 

081 

-009 

-002 

-082 

075 

-159 

027 

05 

Subject  V4 

-450 

072 

-118 

-226 

172 

-050 

135 

168 

-244 

043 

-049 

-029 

-087 

-071 

-123 

06 

Subject  V5 

043 

004 

-283 

-368 

046 

-120 

114 

046 

126 

082 

-122 

-223 

-066 

-048 

-109 

07 

Subject  V6 

-031 

096 

157 

-219 

-074 

-249 

549 

093 

228 

090 

-061 

-052 

017 

-024 

-102 

08 

Subject  V7 

-042 

102 

150 

-400 

-038 

003 

-073 

-138 

009 

138 

-120 

-100 

000 

006 

032 

09 

Subject  Ml 

-254 

-028 

-063 

272 

003 

054 

-090 

079 

250 

-078 

-064 

078 

016 

-065 

-248 

10 

Subject  M2 

-084 

074 

-133 

292 

-138 

016 

-497 

-121 

163 

-015 

-159 

195 

-025 

-091 

-138 

11 

Subject  M3 

-201 

035 

-088 

575 

151 

056 

049 

-101 

-006 

118 

-045 

180 

018 

-203 

-024 

12 

Subject  M4 

-589 

101 

-355 

307 

-521 

028 

-263 

-138 

047 

181 

-008 

126 

012 

-066 

-249 

13 

Subject  M5 

-270 

-172 

-012 

434 

-043 

021 

on 

-201 

050 

-121 

-136 

233 

-065 

-070 

-326 

14 

Order  1 

-242 

-190 

-132 

-068 

-061 

-027 

012 

-133 

-037 

-143 

029 

-068 

131 

-116 

045 

15 

Order  2 

-202 

021 

-074 

-100 

049 

147 

-029 

124 

-215 

-035 

066 

204 

-049 

173 

-056 

16 

Order  3 

-071 

-022 

-057 

-056 

037 

-058 

005 

-on 

-129 

027 

-073 

137 

120 

099 

-154 

17 

Order  4 

-061 

-043 

045 

-094 

002 

-081 

062 

-093 

-087 

010 

072 

169 

052 

-098 

-096 

18 

Order  5 

-074 

-046 

-073 

-015 

-019 

115 

-004 

-033 

-209 

149 

019 

178 

047 

123 

-173 

19 

Order  6 

007 

-056 

on 

-155 

064 

105 

-044 

-014 

-108 

017 

-016 

146 

111 

172 

-073 

20 

Period  1 

026 

306 

-213 

170 

-120 

-074 

-072 

020 

119 

-027 

-105 

338 

-017 

185 

-341 

21 

Condition  1 

-037 

148 

-071 

135 

059 

-106 

-028 

-002 

-047 

050 

-154 

-190 

138 

190 

058 

22 

Condition  2 

048 

118 

-199 

223 

-048 

-107 

014 

043 

050 

106 

255 

-235 

-027 

-186 

-136 

23 

Condition  3 

-444 

-103 

671 

202 

-019 

030 

-075 

151 

083 

112 

-165 

-088 

-094 

068 

132 

24 

Condition  4 

090 

070 

022 

103 

-054 

-085 

-027 

-177 

-056 

-on 

220 

-083 

-033 

-022 

147 

25 

Condition  5 

-000 

118 

051 

-115 

018 

-258 

031 

-085 

130 

077 

185 

194 

-102 

167 

-119 

26 

Condition  6 

098 

013 

-101 

009 

050 

-144 

054 

-012 

009 

132 

291 

-116 

-194 

-167 

091 

27 

PI  xCl 

029 

-042 

050 

-042 

034 

015 

049 

017 

066 

021 

037 

027 

-205 

-049 

083 

28 

PI  xC2 

010 

-163 

207 

-217 

115 

on 

069 

-178 

062 

035 

-017 

161 

-024 

102 

-028 

29 

PI  xC3 

322 

021 

-460 

-283 

047 

-148 

147 

-134 

142 

053 

089 

-036 

-166 

-068 

000 

30 

PI  xC4 

-027 

021 

126 

-156 

053 

-013 

090 

130 

068 

075 

115 

-004 

-063 

-022 

-251 

31 

PI  xC5 

-029 

-033 

no 

-043 

094 

-138 

-002 

-155 

-099 

-098 

-134 

-294 

116 

-041 

022 

32 

PI  xC6 

-000 

-070 

124 

022 

-036 

-036 

-035 

-018 

019 

-068 

033 

-068 

010 

068 

-083 

33 

PI  xOl 

-096 

185 

188 

-015 

-105 

058 

-084 

-256 

-066 

001 

029 

-026 

140 

038 

112 

34 

PI  x02 

-026 

-004 

004 

044 

-018 

143 

044 

022 

061 

010 

-099 

-236 

085 

-181 

299 

35 

PI  x03 

-034 

100 

016 

151 

-053 

126 

-029 

115 

-038 

-131 

078 

-222 

-088 

-079 

183 

36 

PI  x04 

-018 

-044 

-058 

056 

002 

222 

-127 

109 

-152 

-006 

-034 

-150 

163 

113 

213 

137 

PI  x05 

-029 

-012 

024 

027 

090 

122 

-008 

088 

-061 

-060 

030 

-120 

115 

-020 

262 

138 

PI  x06 

-049 

001 

001 

056 

-065 

037 

019 

086 

045 

028 

003 

-051 

010 

-187 

231 

Note:  three  decimal  points  omitted. 
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The  B-weights  for  predicting  each  RDF  (using  the  predictor  variables)  and  their  levels  of 
significance  are  shown  in  Table  13.  Conversion  from  p- weights  to  B-weights  is  based  on  the 
earlier  discussed  equation  where  B. .  =  p. .  svi  /  sx^.  Here,  is  RDF  i's  standard  deviation 
(assumed  to  equal  1.0)  and  sx^  is  predictor  j's  standard  deviation  (shown  in  Table  1). 


Table  13.  [  B'cx]:  Raw-Score  B-Weights  for  Predicting  RDFs 
Rotated  Diagonal  Factors  (RDFs) 

X  Variables  Ft  F2  F3  F4  F5  F6  F7  F8  F9  FIO  Fit  F12  F13  F14  F15 

Verb-RespMd.  -.169  .035  -.216  2.118  .165  .139  -2.28  -.313  .417  -.070  .145  .838  -.086  -.345  -.428 

Subject  VI  -1.66-  1.723 -.647  -.509  .534  -.606  .418  .495  1.105  .150  -.494  -.111  .007  .468  -.523 

Subject  V2  -.564  .098  .654  .015  .144  -.182  .484  .050  .186  .348  .012  -.340  -.495  -.078  -.362 

Subject  V3  -1.88  1.14  .033  -1.34  .897  -.256  .618  .207  .313  -.031  -.006  -.322  .292  -.622  .109 

Subject V4  -1.75  .278  -.457  -.879  .666  -.192  .521  .652  -.947  .170  -.190  -.115  -.335  -.283  -.472 

Subject  V5  .168  .014  -1.10  -1.43  .176  -.464  .442  .182  .487  .321  -.472  -.871  -.251  -.193  -.419 

Subject  V6  -.119  .374  .608  -.854  -.287  -.961  2.13  .362  .882  .353  -.239  -.207  .070  -.099  -.392 

SubjectV7  -.164  .395  .580  -1.56  -.153  .017  -.285  -.535  .034  .538  -.463  -.390  .002  .018  .132 

Subject  Ml  -.993  -.104  -.247  1.06  .017  .206  -.349  .306  .971  -.309  -.251  .301  .064  -.250  -.962 

Subject  M2  -.332  .294  -.521  1.14  -.530  .060  -1.93  -.470  .  632  -.065  -.621  .755  -.097  -.349  -.534 

Subject  M3  -.786  .138  -.341  2.24  .  589  .215  .193  -.392  -.023  .453  -.177  .  696  .072  -.786  -.091 

Subject  M4  -2.29  .395  -1.38  1.19  -2.02  .103  -1.02  -.536  .183  .697  -.035  .489  .047  -.252  -.964 

Subject  M5  -1.06  -.667  -.048  1.69  -.163  .078  .043  -.782  .193  -.474  -.528  .902  -.251  -.266  -1.26 

Order  1  -.701  -.531  -.386  -.179  -.178  -.086  .026  -.387  -.100  -.415  .068  -.201  .389  -.329  .125 

Order2  -.589  .069  -.212  -.270  .135  .411  -.087  .350  -.613  -.102  .179  .575  -.124  .499  -.164 

Order  3  -.212  -.052  -.170  -.143  .103  -.172  .005  -.037  -.369  .070  -.219  .388  .360  .277  -.441 

Order 4  -.183  -.111  .122  -.251  .003  -.240  .171  -.272  -.244  .026  .195  .475  .161  -.282  -.275 

Order  5  -.221  -.120  -.215  -.026  -.058  .321  -.019  -.096  -.594  .417  .038  .506  .148  .351  -.498 

Order  6  .011  -.152  .023  -.432  .183  .297  -.129  -.044  -.315  .046  -.061  .421  .327  .493  -.213 

Period  1  .045  .613  -.438  .334  -.226  -.149  -.147  .036  .213  -.046  -.222  .693  -.041  .369  -.699 

Condition  1  -.100  .414  -.206  .373  .184  -.296  -.077  -.005  -.157  .149  -.439  -.532  .381  .545  .164 

Condition  2  .144  .330  -.570  .619  -.125  -.299  .052  .124  .122  .309  .734  -.661  -.093  -.528  -.397 

Condition  3  -1.26  -.304  1.92  .563  -.040  .094  -.206  .437  .215  .331  -.472  -.235  -.287  .198  .369 

Condition  4  .265  .194  .061  .278  -.138  -.239  -.071  -.501  -.184  -.023  .635  -.224  -.110  -.063  .414 

Condition  5  .002  .330  .140  -.342  .069  -.731  .094  -.239  .349  .224  .533  .570  -.314  .473  -.344 

Condition  6  .283  .026  -.291  .006  .158  -.405  .164  -.034  .008  .387  .837  -.321  -.573  -.477  .258 

PlxCl  .105  -.158  .199  -.144  .118  .046  .191  .075  .288  .066  .139  .082  -.772  -.195  .332 

PlxC2  .028  -.626  .805  -.821  .436  .031  .256  -.684  .270  .122  -.078  .604  -.068  .395  -.095 

PlxC3  1.24  .096  -1.79-1.083  .164  -.592  .564  -.516  .583  .188  .339  -.167  -.616  -.266  .020 

PlxC4  -.114  .087  .492  -.586  .189  -.059  .344  .505  .295  .276  .431  -.044  -.216  -.083  -.964 

PlxC5  -.119  -.120  .434  -.150  .345  -.551  -.013  -.598  -.353  -.390  -.534-1.170  .488  -.155  .095 

PlxC6  -.009  -.262  .483  .107  -.159  -.155  -.145  -.060  .099  -.282  .113  -.285  .068  .267  -.315 

PlxOl  -.358  .704  .742  -.070  -.406  .242  -.319  -.991  -.265  .011  .140  -.091  .518  .147  .442 

Plx02  -.086  -.024  .020  .159  -.066  .573  .174  .084  .234  .042  -.361  -.909  .303  -.707  1.168 

Plx03  -.116  .378  .075  .573  -.203  .502  -.098  .446  -.147  -.501  .330  -.863  -.367  -.297  .715 

Plx04  -.056  -.184  -.215  .201  .011  .878  -.486  .423  -.597  -.018  -.110  -.573  .615  .442  .827 

Plx05  -.097  -.054  .103  .087  .349  .491  -.018  .336  -.240  -.225  .144  -.466  .423  -.076  1.025 

Plx06  -.180  -.004  .016  .207  -.254  .149  .079  .339  .185  .108  .034  -.207  .029  -.724  .909 

Constant  1.239  -.373  .420-1.352  -.052  .218  1.261  .340  -.388  -.250  -.056  -.698  .065  .129  .850 

Notes:  Bold  =  significance  <  .001;  italics  =  significance  <  .01. 
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Submatrix  [  Ucc  ],  which  contains  the  RDFs  variance  that  was  unexplained  by  the  PDFs, 
is  shown  in  Table  14.  Table  15  shows  a  portion  of  the  inverse  of  [  Rxx  ]•  These  submatrices  are 
shown  because  they  contain  the  additional  information  used  in  determining: 

SpYjXi  =  (  -  Ixi  Xi  (1  -  R^Y;j.l  ...  m)  /  (N  -  m  -  1)  . 

where  Ixi  xi  is  the  value  found  in  the  ith  diagonal  cell  (i.e.,  row  i  and  column  i)  of  the  inverse 
matrix  [  R''xx]  and  (1  -  R^y^  .i  ...  m)  is  the  value  found  in  the  jth  diagonal  cell  of  the  [Uyy]  matrix. 
The  t-test  of  significance  (with  N-m-1  dfs)  for  each  x  is  t^Y^Xi  =  P  Yjxi  /  s^y^Xi 


Table  14.  [  Ucc  ]•  The  RDF  Variance  Unexplained  by  the  PDFs 


Vb 

Variable 

FI 

F2 

F3 

F4 

F5 

F6 

F7 

F8 

F9 

FIO 

Fll 

F12 

F13 

F14 

F15 

39 

RDF 

1 

173 

-049 

036 

057 

-050 

-010 

-045 

-002 

-025 

-020 

-057 

031 

050 

-022 

-046 

40 

RDF 

2 

-049 

433 

022 

095 

056 

-005 

062 

064 

084 

-003 

-046 

025 

-037 

057 

-038 

41 

RDF 

3 

036 

022 

332 

-017 

-090 

007 

-058 

052 

-030 

045 

047 

-028 

-002 

015 

-107 

42 

RDF 

4 

057 

095 

-017 

483 

-033 

-047 

-001 

-042 

009 

-008 

014 

-068 

042 

042 

063 

43 

RDF 

5 

-050 

056 

-090 

-033 

425 

027 

093 

-167 

001 

046 

-007 

-017 

030 

036 

-066 

44 

RDF 

6 

-010 

-005 

007 

-047 

027 

794 

009 

-063 

112 

-031 

007 

-039 

013 

-004 

-053 

45 

RDF 

7 

-045 

062 

-058 

-001 

093 

009 

116 

-044 

064 

034 

-005 

018 

-040 

006 

018 

46 

RDF 

8 

-002 

064 

052 

-042 

-167 

-063 

-044 

607 

-033 

-044 

017 

-014 

096 

-027 

on 

47 

RDF 

9 

-025 

084 

-030 

009 

001 

112 

064 

-033 

617 

026 

077 

-001 

012 

-019 

045 

48 

RDF 

10 

-020 

-003 

045 

-008 

046 

-031 

034 

-044 

026 

815 

-038 

-019 

033 

016 

-008 

49 

RDF 

11 

-057 

-046 

047 

014 

-007 

007 

-005 

017 

077 

-038 

654 

-018 

030 

109 

065 

50 

RDF 

12 

031 

025 

-028 

-068 

-017 

-039 

018 

-014 

-001 

-019 

-018 

761 

035 

-049 

077 

51 

RDF 

13 

050 

-037 

-002 

042 

030 

013 

-040 

096 

012 

033 

030 

035 

824 

-010 

-023 

52 

RDF 

14 

-022 

057 

015 

042 

036 

-004 

006 

-027 

-019 

016 

109 

-049 

-010 

773 

033 

53 

RDF 

15 

-046 

-038 

-107 

063 

-066 

-053 

018 

Oil 

045 

-008 

065 

077 

-023 

033 

703 
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Table  15.  A  Portion  of  Submatrix  [  R’'xx 

1 

Vb  Variable  XI 

X2 

X3 

X4  X5 

X6  X7 

X8 

X9 

XIO 

Xll 

X12 

X13 

X14 

X15 

01 

Verb-Resp  -6857 

1784 

1784 

1784  1784 

1784  1784 

1784 

-1784 

-1784 

-1784 

-1784 

-1784 

000 

000 

02 

Subject  VI  1784 

-1857 

-929 

-929  -929 

-929  -929 

-929 

000 

000 

000 

000 

000 

000 

000 

03 

Subject  V2  1784 

-929 

-1857 

-929  -929 

-929  -929 

-929 

000 

000 

000 

000 

000 

000 

000 

04 

Subject  V3  1784 

-929 

-929 

-1857  -929 

-929  -929 

-929 

000 

000 

000 

000 

000 

000 

000 

05 

Subject  V4  1784 

-929 

-929 

-929  -1857 

-929  -929 

-929 

000 

000 

000 

000 

000 

000 

000 

06 

Subject  V5  1784 

-929 

-929 

-929  -929 

-1857  -929 

-929 

000 

000 

000 

000 

000 

000 

000 

07 

Subject  V6  1784 

-929 

-929 

-929  -929 

-929  -1857 

-929 

000 

000 

000 

000 

000 

000 

000 

08 

Subject  V7  1784 

-929 

-929 

-929  -929 

-929  -929 

-1857 

000 

000 

000 

000 

000 

000 

000 

09 

Subject  Ml  -1784 

000 

000 

000  000 

000  000 

000 

-1857 

-929 

-929 

-929 

-929 

000 

000 

10 

Subject  M2 -1784 

000 

000 

000  000 

000-  000 

000 

-929 

-1857 

-929 

-929 

-929 

000 

000 

11 

Subject  M3  -1784 

000 

000 

000  000 

000  000 

000 

-929 

-929 

-1857 

-929 

-929 

000 

000 

12 

Subject  M4 -1784 

000 

000 

000  000 

000  000 

000 

-929 

-929 

-929 

-1857 

-929 

000 

000 

13 

Subject  M5 -1784 

000 

000 

000  000 

000  000 

000 

-929 

-929 

-929 

-929 

-1857 

000 

000 

14 

Order  1  000 

000 

000 

000  000 

000  000 

000 

000 

000 

000 

000 

000 

-1857 

-2814 

15 

Order  2  000 

000 

000 

000  000 

000  000 

000 

000 

000 

000 

000 

000 

-2814 

-1857 

Note:  three  decimal  points  omitted;  bold  indicates  diagonal  entries. 


Finally,  it  has  been  indicated  that  one  of  the  purposes  of  the  RDF  approach  is  to  be  able  to 
partition  the  variance  into  that  which  is  attributable  to  various  independent  effects  (such  as 
individual  differences  and  experimental  manipulations)  and  that  which  is  attributable  to  in 
between-tasks,  within-task,  and  within-criterion  effects. 

Recalling  that  the  RDFs  are  independent  from  one  another,  it  is  possible  to  sum  the 
portions  of  the  effects'  variance  for  a  specific  type  of  factors  (i.e.,  between-tasks,  within-task,  and 
within-criterion).  The  data  shown  in  Table  16  were  derived  from  Table  11  (i.e.,  "The  Source 
Table  for  the  RDFs").  The  variance  explained  (i.e.,  sums  of  squares  (SSs))  for  each  effect  were 
summed  separately  for  the  three  factor  types.  The  SSs  were  then  divided  by  their  appropriate 
degrees  of  freedom  to  find  the  mean  squares  (MSs).  Finally,  the  MSs  were  divided  by  the 
residual  MS  to  compute  F-values. 
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At  the  far  right  of  Table  16,  another  second  of  showing  the  impact  of  the  various  effects 
on  the  different  factor  types  is  provided.  Here,  the  SSs  for  a  particular  factor  type  were  divided 
by  "15"  (i.e.,  the  total  amount  of  all  criterion  variance  across  the  15  RDFs).  This  shows,  by 
factor  type,  the  percent  of  all  RDF  variance  attributable  to  the  various  effects.  For  example,  the 
"Subjects"  effect  accounted  for  a  total  of  13.78  percent  (i.e.,  =  7.27  +  1.00  +  5.51)  of  all  RDF 
variance. 


Table  16.  Source  Table  by  Factor  Types 


between-tasks  within-task  within-criterion  Percent  of  all  RDF 


Effect 

dfs 

SSs 

MSs 

F 

SSs  MSs 

F 

SSs  MSs 

F 

BT 

WT 

wc 

Sum 

Verb-Resp 

1 

.372 

.3720  206.67 

.175  .1750 

11.29 

.058  .0580 

1.58 

2.48 

1.17 

.39 

4.04 

Subjects 

12 

1.091 

.0909 

50.44 

.150  .0125 

.81 

.826  .0688 

1.87 

7.27 

1.00 

5.51 

13.78 

Orders 

6 

.104 

.0173 

9.67 

.165  .0275 

1.77 

.371  .0618 

1.68 

.69 

1.10 

2.47 

4.26 

Periods 

1 

.003 

.0030 

1.67 

.112  .1120 

7.23 

.041  .0410 

1.12 

.02 

.75 

.27 

1.04 

Conditions 

6 

.073 

.0122 

6.78 

.270  .0691 

2.90 

.519  .0865 

2.36 

.49 

1.80 

3.46 

5.75 

PxC 

6 

.055 

.0092 

5.11 

.240  .0400 

2.58 

.242  .0403 

1.10 

.37 

1.60 

1.61 

3.58 

PxO 

6 

.012 

.0020 

1.11 

.172  .0287 

.77 

.188  .0313 

.85 

.08 

1.15 

1.25 

2.48 

Residual 

157 

.290  .0018 

2.466  .0157 

5.755  .0367 

1.93 

16.44  38.37 

56.74 

Explained 

38 

1.710 

2.534 

2.245 

11.40 

16.89 

14.97 

43.26 

Total 

195 

2.000 

5.000 

8.000 

13.33  33.33 

53.33 

100.00 

Notes:  a  bold F  was  significant p<  .01;  an  underlined  Fwas  significant p<  .05. 


The  interpretation  of  each  of  the  between-tasks,  within-tasks,  and  within-criterion  RDFs 
is  discussed  in  Section  4. 
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4.  INTERPRETATION  OF  THE  FACTORS 

A  major  purpose  of  the  RDF  approach  is  to  gain  insight  into  how  performance  on 
multiple  tasks  and  multiple  criteria  are  effected  by  both  individual  differences  as  well  as  by 
experimental  manipulations.  Each  RDF  now  represents  an  independent  composite  of  criterion 
performance  variance  and  may  be  one  of  three  theoretical  types:  (a)  a  between-tasks  type  causing 
Ss  performance  on  different  tasks  to  covary  together,  (b)  a  within-task  type  causing  Ss' 
performance  on  different  criteria  for  a  specific  task  to  covary  together  (but  not  influencing 
performance  on  any  other  task),  and  (c)  a  within-criterion  type  causing  Ss  to  perform 
differentially  well  on  a  single  criterion  of  one  task  (but  not  influencing  behavior  on  any  other 
criterion  for  that  task  or  any  other  task).  The  predictor  and  criterion  loadings  on  each  RDF  show 
the  relationship  of  that  variable  with  that  factor.  The  square  of  these  loadings  shows  that 
variable's  portion  of  variance,  which  is  accounted  for  by  that  factor.  Because  all  of  the  variance 
of  all  of  the  criteria  are  being  explained  by  the  15  rotated  diagonal  factors,  they  provide  an 
alternative  and  more  meaningful  explanation  of  the  overall  criterion  variance. 

In  the  following  subsections,  each  RDF  will  be  discussed,  first,  in  terms  of  how  the 
various  criteria  relate  to  each  other  on  that  faetor.  This  interpretation  establishes  the  type  of 
factor  it  appears  to  be  (i.e.,  between-tasks,  within-task,  or  within-criterion).  Interpretation  of  the 
nature  of  an  RDF  can  be  based  on  both  the  criteria  significantly  related  to  it  (as  seen  in  Table  7) 
and  its  significant  raw-score  beta  weights  (as  seen  in  Table  13).  The  B-weights  show  us  how 
individual  differences  and  experimental  manipulations  significantly  effect  scores  for  each  factor. 
With  regard  to  individual  differences.  Table  13  shows  that  11  of  the  15  factors  had  at  least  one 
subject  variable  with  a  significant  (p<  .001)  raw-score  B-weight  on  it.  Of  the  remaining  four 
factors,  three  of  them  had  at  least  one  subject  variable  with  a  significant  (p<  .01)  raw-score 
B-weight.  Thus,  only  one  factor  (i.e.,  RDF  13)  failed  to  show  a  significant  impact  due  to 
individual  differences. 
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4.1  Adjusted  Weights  for  Individual  and  Group  Differences 

While  the  B-weights  are  useful  in  indicating  the  relative  effects  of  levels  of  subject 
effects,  there  is  another  way  to  examine  the  effect  of  individual  differences.  Before  proceeding 
to  the  interpretation  of  the  factors,  it  is  useful  to  develop  what  can  be  referred  to  as  adjusted 
weights.  The  purpose  of  adjusting  the  B-weights  is  to  examine  how  each  level  of  every  effect 
contributes  to  the  predicted  RDF  scores.  It  may  be  recalled  that  there  were  fourteen  Ss  in  this 
study  and  eight  of  them  used  the  "Verbal  Response"  mode  during  the  Com  tasks  and  six  of  them 
used  the  "Manual  Response"  mode.  One  dichotomous  variable  was  created  to  represent  this 
group  variable.  It  may  also  be  recalled  that  only  seven  dichotomous  variables  were  created  to 
represent  seven  of  the  eight  Ss  in  the  verbal  response  group,  and  that  only  five  dichotomous 
variables  were  created  to  represent  five  of  the  six  Ss  in  the  manual  response  group.  The  choice 
of  which  seven  and  which  five  were  represented  as  variables  was  arbitrary.  From  a  prediction 
standpoint,  it  does  not  affect  the  analysis.  However,  when  the  ^-weights  were  converted  to 
B-weights,  there  was  also  a  constant  to  be  added  (see  the  last  row  in  Table  13).  Sinee  neither 
Subject  V8  nor  Subject  M6  were  represented  by  a  separate  dichotomous  predictor  variable,  their 
B-weights  are  effectively  zero.  However,  Subject  Mb's  predicted  factor  score  on  any  RDF 
included  the  constant  for  that  RDF.  Further,  Subject  V8  was  a  member  of  the  "Verbal-Response" 
group,  and  his  predicted  factor  score  for  any  RDF  included  the  B-weight  for  Verbal-Response  as 
well  as  the  constant  for  that  factor.  Table  17  shows  the  result  of  the  predicted  factors  scores  for 
all  Ss  based  only  on  their  Subject  B-weights,  the  Verbal -Response  B-weight,  and  the  constant  for 
each  RDF. 
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Variable 

Table  17.  Predicted  RDF  Factor  Scores  Based  on  Subject  Bs, 
the  Verbal-Response  B  (When  Applicable),  and  the  Constant 

FI  F2  F3  F4  F5  F6  F7  F8  F9  FIO  Fll  F12  F13  FI4 

F15 

Subject  VI 

-593  -2062 

-450  255  657 

-249 

-603 

520 

1115  -167  -413  036  -017  252 

-108 

Subject  V2 

503  -238 

852  781  267 

173 

-538 

077 

199  031  092  -191  -519  -293 

050 

Subject  V3 

-809  804 

228  -574  1021 

101 

-404 

232 

325  -351  074  -173  267  -837 

523 

Subject  V4 

-683  -058 

-262  -113  790 

165 

-502 

679 

-936  -149  -no  032  -360  -498 

-058 

Subject  V5 

1235  -321 

-904  -665  300 

-109 

-581 

207 

499  002  -391  -721  -277  -408 

-004 

Subject  V6 

946  036 

805  -088  -165 

-606 

1106 

387 

895  035  -157  -058  044  -315 

021 

Subject  V7 

903  058 

776  -791  -028 

371 

-1309 

-507 

044  219  -384  -242  -022  -199 

544 

Subject  V8 

1066  -338 

194  763  121 

359 

-1025 

027 

010  -316  081  145  -022  -221 

419 

Subject  Ml 

244  -476 

167  -293  -025 

421 

912 

646 

567  -556  -315  -389  127  -120 

-119 

Subject  M2 

903  -080 

-107  -213  -573 

277 

-667 

-133 

228  -311  -683  065  -035  -221 

307 

Subject  M3 

449  -235 

070  883  549 

432 

1453 

-053 

-428  205  -239  007  134  -657 

750 

Subject  M4 

-1058  025 

-969  -156  -2058 

320 

241 

-198 

-223  450  -099  -202  109  -123 

-122 

Subject  M5 

182  -1039 

365  335  -205 

295 

1305 

-443 

-212  -722  -593  213  -190  -138 

-422 

Subject  M6 

1231  -369 

410  -1349  -037 

213 

1262 

339 

-405  -252  -066  -693  063  133 

843 

Group  Differences 

MeanV-R  321  -265 

155  -054  370 

026 

-482 

203 

269  -087  -151  -147  -113  -315 

173 

Mean  M-R 

325  -362 

-on  -132  -391 

326 

751 

026 

-079  -198  -333  -166  035  -188 

206 

Mn  V  -  Mn  M 

-004  097 

166  078  762 

-301 

-1233 

176 

348  111  182  020  -148  -127 

-033 

Note:  Three  decimal  points  omitted.  Bold  indicates  significance  p<  .001. 


Mean  differences  of  the  two  groups  for  RDFs  are  attributable  to  either  (a)  differences  in 
"capability"  of  Ss  within  each  group  or  (b)  differences  attributable  to  the  "mode  of  response"  Ss 
used  in  the  Com  task.  It  has  already  been  seen  that  RDFs  5,  7,  and  9  were  significantly  different 
(p<  .001  level)  from  what  would  be  expected  by  chance  alone  (if  assignment  to  those  groups 
actually  had  no  effect).  This  conclusion  is  also  supported  by  the  fact  that  those  RDFs  had  the 
largest  differences  between  the  means  of  the  two  groups  (as  shown  at  the  bottom  of  Table  17). 


The  two  groups  discussed  above  could  be  statistically  equated  by  subtracting  the  mean  of 
each  group  from  the  predicted  scores  for  each  S  in  that  group.  When  this  computation  is 
performed,  the  average  of  the  adjusted  subject  weights  will  be  zero,  and  the  difference  between 
the  means  of  the  two  groups  (shown  in  Table  17)  must  now  be  used  as  the  "Verb-Resp"  effect. 
Finally,  the  mean  of  the  manual-response  group  must  be  used  as  a  "constant"  to  be  added  for 
each  predicted  factor  score.  The  predictor  adjusted  weights  for  the  subjects  and  group  effects  are 
shown  in  Table  1 8. 
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Table  18.  Group  and  Subject  Adjusted  Weights  for  Predicting  RDFs 

Variable 

Fl 

F2 

F3  F4 

F5 

F6  F7 

F8  F9  FIO 

Fll  F12 

F13 

F14  F15 

Verb-Resp 

-004 

097 

166  078 

762 

-301  -1233 

176  348  111 

182  020 

-148 

-127  -033 

Subject  VI 

-914 

-1797 

-605  309 

287 

-275  -121 

317  846  -080 

-262  183 

096 

567  -281 

Subject  V2 

182 

027 

697  835 

-103 

147  -056 

-126  -070  118 

243  -044 

-406 

022  -123 

Subject  V3 

-1130 

1069 

073  -520 

651 

075  078 

029  056  -264 

225  -026 

380 

-522  350 

Subject  V4 

-1004 

207 

-417  -059 

420 

139  -020 

476  -1205  -062 

041  179 

-247 

-183  -231 

Subject  V5 

914 

-056 

-1059  -611 

-070 

-135  -099 

004  230  089 

-240  -574 

-164 

-093  -177 

Subject  V6 

625 

301 

650  -034 

-535 

-632  1588 

184  626  122 

-006  089 

157 

000  -152 

Subject  V7 

582 

323 

621  -737 

-398 

345  -827 

-710  -225  306 

-233  -095 

091 

116  371 

Subject  V8 

745 

-073 

039  817 

-249 

333  -543 

-176  -259  -229 

232  292 

091 

094  246 

Subject  Ml 

-081 

-114 

178  -161 

366 

095  161 

620  646  -358 

018  -223 

092 

068  -325 

Subject  M2 

578 

282 

-096  -081 

-182 

-049  -1418 

-159  307  -113 

-350  231 

-070 

-033  101 

Subject  M3 

124 

127 

081  1015 

940 

106  702 

-079  -349  403 

094  173 

099 

-469  544 

Subject  M4 

-1383 

387 

-958  -024 

-1667 

-006  -510 

-224  -144  648 

234  -036 

074 

065  -328 

Subject  M5 

-143 

-677 

376  467 

186 

-031  554 

-469  -133  -524 

-260  379 

-225 

050  -628 

Subject  M6 

906 

-007 

421  -1217 

354 

-113  511 

313  -326  -054 

267  -527 

028 

321  637 

Constants 

325 

-362 

-Oil  -132 

-391 

326  751 

026  -079  -198 

-333  -166 

035 

-188  206 

Note:  underlined  S  scores  show  max.  and  min.  values  within  that  group. 


4.2  Adjusted  Weights  for  Experimental  Effects 


In  the  preceding  subsection,  it  was  shown  that,  even  though  all  Ss  were  not  represented 
with  separate  predictor  dichotomous  variables,  it  was  still  possible  to  determine  their  relative 
position  on  the  RDF  dimension.  Likewise,  "Order  7,"  "Condition  7,"  "Period  3,"  and  all  cells 
representing  interactions  of  these  levels  were  not  represented  by  separate  predictor  variables.  As 
with  the  Ss,  relative  locations  on  those  levels  of  main  effects  and  interactions  on  each  RDF  can 
be  found.  First,  zero  values  for  B-weights  for  non-represented  levels  of  main  effects  and 
interactions  were  assumed.  Next,  predicted  RDF  scores  were  developed  by  applying  all  the 
relevant  B-weights  (i.e..  Order,  Period,  Condition,  PxC,  and  PxO)  for  each  cell.  The  mean  of  the 
predicted  RDF  score  cell  values  was  then  subtracted  from  each  predicted  cell  value.  Means  for 
each  Order,  Period,  and  Condition  were  found  and  subtracted  from  the  predicted  cell  values. 

Table  19  shows  the  adjusted  weights  found  by  a  procedure  similar  to  that  discussed 

for  Ss. 
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Table  19.  Experimental  Manipulation  Adjusted  Weights  for  Predicting  RDFs 

Variable 

FI  F2  F3  F4 

F5 

F6 

F7 

F8  F9 

FIO 

Fll 

F12 

F13 

F14 

F15 

Order  1 

-180  -065  -012  -028 

-113 

-077 

-017 

-249  057 

-134 

029 

-133 

112 

-127 

084 

Order  2 

-105  052  -046  -031 

040 

140 

006 

147  -057 

-021 

002 

034 

-096 

052 

076 

Order  3 

022  061  -023  067 

on 

-074 

004 

059  -021 

-030 

-049 

-026 

-013 

029 

-080 

Order  4 

040  -032  042  -019 

003 

-049 

012 

-026  -036 

016 

040 

042 

044 

-072 

-007 

Order  5 

021  -019  -035  046 

026 

097 

005 

023  -112 

128 

019 

066 

015 

083 

-059 

Order  6 

092  -028  047  -082 

034 

040 

-025 

034  040 

033 

-027 

068 

027 

066 

003 

Order  7 

110  033  026  047 

000 

-079 

013 

012  127 

009 

-015 

-052 

-089 

-030 

-017 

Period  1 

017  149  -082  056 

-048 

-009 

-027 

-003  063 

-024 

-043 

096 

-002 

072 

-099 

Period  3 

-017  -149  082  -056 

048 

009 

027 

003  -063 

024 

043 

-096 

002 

-072 

099 

Condition  1 

-009  095  -112  086 

053 

020 

-021 

043  -056 

-006 

-235 

-087 

104 

156 

096 

Condition  2 

066  -001  -152  084 

-012 

012 

027 

-017  037 

050 

149 

-066 

029 

-126 

-176 

Condition  3 

-270  -125  375  033 

-019 

075 

-019 

121  112 

072 

-220 

-017 

-109 

025 

129 

Condition  4 

090  048  019  -003 

-051 

027 

000 

-075  -064 

-040 

178 

004 

004 

-042 

018 

Condition  5 

-001  069  040  -164 

042 

-208 

012 

-126  038 

-039 

019 

136 

024 

137 

-111 

Condition  6 

111  -054  -105  -008 

009 

-043 

019 

016  -024 

031 

208 

-061 

-121 

-142 

046 

Condition  7 

013  -032  -066  -028 

-023 

119 

-018 

037  -042 

-067 

-099 

089 

068 

-009 

-003 

PI  xCl 

-007  -001  013  031 

-005 

030 

002 

034  015 

010 

009 

029 

-079 

-026 

063 

PI  xC2 

-017  -067  100  -059 

037 

023 

008 

-071  on 

010 

-015 

095 

012 

067 

-015 

PI  xC3 

139  030  -242  -090 

001 

-051 

051 

-042  053 

026 

035 

-003 

-059 

-036 

022 

PI  xC4 

-035  030  051  -026 

004 

016 

023 

090  016 

037 

048 

013 

-008 

-013 

-104 

PI  xC5 

-036  003  043  030 

025 

-046 

-023 

-052  -068 

-049 

-076 

-132 

082 

-022 

033 

PI  xC6 

-022  -015  050  063 

-040 

005 

-040 

016  -009 

-034 

007 

-019 

029 

032 

-020 

PI  xC7 

-022  020  -012  052 

-022 

023 

-022 

025  -018 

-000 

-009 

015 

024 

-002 

022 

P3xCl 

007  001  -013  -031 

005 

-030 

-002 

-034  -015 

-010 

-009 

-029 

079 

026 

-063 

P3xC2 

017  067  -100  059 

-037 

-023 

-008 

071  -on 

-010 

015 

-095 

-012 

-067 

015 

P3xC3 

-139  -030  242  090 

-001 

051 

-051 

042  -053 

-026 

-035 

003 

059 

036 

-022 

P3xC4 

035  -030  -051  026 

-004 

-016 

-023 

-090  -016 

-037 

-048 

-013 

008 

013 

104 

P3xC5 

036  -003  -043  -030 

-025 

046 

023 

052  068 

049 

076 

132 

-082 

022 

-033 

P3xC6 

022  015  -050  -063 

040 

-005 

040 

-016  009 

034 

-007 

019 

-029 

-032 

020 

P3xC7 

022  -020  012  -052 

022 

-023 

022 

-025  018 

000 

009 

-015 

-024 

002 

-022 

PI  xOl 

-030  077  080  -030 

-042 

-021 

-028 

-139  -018 

013 

014 

045 

040 

039 

-034 

PI  x02 

005  -017  -012  -000 

001 

022 

036 

000  046 

017 

-050 

-060 

012 

-071 

060 

PI  x03 

001  035  -006  053 

-016 

013 

-001 

047  -004 

-053 

038 

-053 

-074 

-020 

002 

PI  x04 

009  -037  -043  006 

on 

061 

-050 

044  -061 

009 

-018 

-017 

051 

076 

017 

PI  x05 

003  -021  -002  -009 

055 

on 

010 

033  -015 

-018 

014 

-002 

027 

010 

041 

PI  x06 

-007  -020  -004  003 

-020 

-037 

019 

025  036 

019 

003 

031 

-025 

-055 

003 

PI  x07 

018  -015  -014  -022 

010 

-050 

014 

-on  015 

012 

-001 

058 

-030 

020 

-090 

P3x01 

030  -077  -080  030 

042 

021 

028 

139  018 

-013 

-014 

-045 

-040 

-039 

034 

P3x02 

-005  017  012  000 

-001 

-022 

-036 

-000  -046 

-017 

050 

060 

-012 

071 

-060 

P3x03 

-001  -035  006  -053 

016 

-013 

001 

-047  004 

053 

-038 

053 

074 

020 

-002 

P3  x04 

-009  037  043  -006 

-on 

-061 

050 

-044  061 

-009 

018 

017 

-051 

-076 

-017 

P3x05 

-003  021  002  009 

-055 

-on 

-010 

-033  015 

018 

-014 

002 

-027 

-010 

-041 

P3x06 

007  020  004  -003 

020 

037 

-019 

-025  -036 

-019 

-003 

-031 

025 

055 

-003 

P3x07 

-018  015  014  022 

-010 

050 

-014 

on  -015 

-012 

001 

-058 

030 

-020 

090 

Constants 

-no  153  -068  066 

-037 

-076 

-031 

-038  -025 

046 

061 

131 

012 

129 

-146 

Note:  Three  decimal  points  omitted  for  all  values.  Bold  indicates  significance  p<  .001. 
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In  Table  19,  the  means  of  the  adjusted  weights  for  the  seven  Orders,  two  Periods,  seven 
Conditions,  and  all  interactions  are  now  zero.  Thus,  each  adjusted  weight  shows  the  relative 
impact  of  each  level  in  terms  of  standard  deviation  units  for  a  scale  having  a  mean  of  zero  and  a 
standard  deviation  of  one.  The  values  of  the  "constants"  for  each  RDF  (shown  at  the  bottom  of 
Table  19)  represent  the  sum  of  all  the  means  subtracted  from  the  predicted  factor  scores. 
Technically,  these  constants  could  be  added  to  the  constants  shown  for  the  adjusted  weights  for 
Ss  shown  in  Table  18.  These  adjusted  weights  and  constants  and  the  adjusted  weights  and 
constant  for  the  Ss  will  produce  exactly  the  same  predicted  factor  scores  that  the  B-weights 
produce.  The  advantages  of  the  adjusted  weights  is  that  they  enable  determining  the  relative 
positions  of  all  levels,  even  those  not  represented  by  separate  predictor  dichotomous  variables. 

4.3  Overview  of  Signiflcant  Experimental  Manipulation  Effects 

The  following  subsections  briefly  discuss  the  significant  experimental  manipulation 
effects  (i.e..  Orders,  Periods,  Conditions,  and  interactions)  studied  and  how  they  were  interpreted 
for  the  various  RDFs. 

4.3.1  Order  Effects 


Only  two  RDFs  had  significant  order  effects  (p<  .001  level).  They  were  RDF  1 
(0|  =  -.180)  and  RDF  8  (Oi  =  -.249).  This  indicates  that  performance  on  whatever  these  RDFs 
are,  was  significantly  worse  for  the  initial  session  than  for  subsequent  sessions.  Improvement 
across  the  various  orders  can  be  interpreted  as  a  learning  or  practice  effect  as  the  study 
progressed. 

4.3.2  Period  Effects 


According  to  the  RDF  Source  Table  (Table  11),  RDF  2  and  RDF  3  had  significant 
(p<  .001)  period  effects.  The  adjusted  weights  for  these  two  RDFs  were:  RDF  2,  Pi  (.149)  and  P3 
(-.149)  and  RDF  3,  P]  (-.082)  and  P3  (.082).  Whatever  these  RDFs  represent,  the  direction  of 
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these  adjusted  weights  indicates  that  Ss,  as  each  session  progressed  (i.e.,  from  P]  to  P3), 
performed  relatively  worse  on  RDF  1  and  relatively  better  on  RDF  3. 

4.3.3  Period  by  Order  Effects 


Table  19  shows  that  none  of  the  PxO  terms  reached  significance  at  the  .001  level.  Indeed, 
most  of  the  adjusted  weights  were  extremely  close  to  zero.  The  largest  PxO  adjusted  weights 
were  for  RDF  8  for  PiOi  (-.139)  and  P3O]  (.139).  Although  not  significant,  these  weights 
indicate  that  Ss'  performance  on  RDF  8  was,  for  whatever  condition  they  underwent  first, 
somewhat  worse  in  period  1  than  in  period  3. 

4.3.4  Condition  and  Period  by  Condition  Effects 


Only  two  RDFs  attained  significant  (p<  .001)  PxC  adjusted  weights.  They  were:  RDF  1, 
P,C3  (.139)  and  RDF  1,  P3C3  (-.139)  and  RDF  3  for  P1C3  (-.242)  and  P3C3  (.242).  These  RDFs 
were  also  the  only  ones  that  showed  significant  condition  effects  and  they  were,  respectively, 
RDF  1;  C3  (-.270)  and  RDF  3;  C3  (.375).  This  finding  suggests  that  Ss,  after  undergoing 
Condition  3  (more  difficult  tracking  during  P2),  exhibit  less  of  whatever  RDF  1  represents  and 
more  of  whatever  RDF  3  represents. 

4.4  RDF  1:  Ability  to  Time-Share  Tasks 

The  four  highest  loadings  on  RDF  1  addressed  (low)  RMS  Error  on  the  tracking  task: 


RMSE  in  Elevation  during  the  TA  task  -.997 

RMSE  in  Azimuth  during  the  TA  task  -.846 

RMSE  in  Elevation  during  the  Com  task  -.848 

RMSE  in  Azimuth  during  the  Com  task  -.976 
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Clearly,  because  the  squares  of  these  loadings  were  so  close  to  one,  this  RDF  explained  most  of 
the  tracking  error  variance  in  both  elevation  and  azimuth  during  the  occasions  in  which  either  TA 
and  Com  tasks  were  being  performed.  The  RDF  also  showed  significant  positive  loadings  for  the 
four  corresponding  Stick  Manipulation  criteria: 


Stick  Manipulation  in  Elevation  during  the  TA  task  .429 

Stick  Manipulation  in  Azimuth  during  the  TA  task  .454 

Stick  Manipulation  in  Elevation  during  the  Com  task  .3 1 7 

Stick  Manipulation  in  Azimuth  during  the  Com  task  .376 


The  opposite  signs  of  loadings  for  RMSE  and  Stick  Manipulations  were  expected  since  Ss  who 
manipulated  their  control  stick  more  frequently  would,  in  general,  be  expected  to  produce  less 
tracking  error.  Because  the  loadings  of  all  RMS  Error  were  negative,  the  RDF  indicates  desired 
tracking  performance.  However,  significant  loadings  for  three  of  the  four  criteria  measuring  TA 
performance  also  loaded  significantly  on  this  RDF: 


TA  Percent  Correct  Responses 

.552 

TA  Percent  Incorrect  Responses 

-.469 

TA  Median  Time  for  Incorrect  Responses 

-.266 

Further,  the  signs  of  these  loadings  also  indicated  desired  tactical  assessment  performance. 
While  the  three  criteria  for  the  Com  task  did  not  reach  significance  and  were  not  nearly  as  high  as 
those  for  tactical  assessment,  the  signs  of  their  non-zero  loadings  indicated  that  the  RDF  was  also 
indicative  of  desired  communications  performance. 

Had  this  RDF  indicated  desired  perfomnance  for  tactical  assessment  and  communications, 
but  undesired  performance  for  tracking,  the  RDF  would  have  indicated  a  trade-off  of  attention 
from  the  tracking  task  to  the  other  tasks.  However,  since  the  loadings  of  some  criteria  for  all 
three  tasks  indicated  desired  performance,  the  results  suggested  that  Ss  who  did  well  in  one  task, 
especially  tracking,  also  did  well  in  the  other  two.  The  fact  that  TA  Median  Time  for  Correct 
Responses  did  not  influence  this  factor  ruled  out  the  possible  interpretation  that  the  factor 
represented  simply  "rapid  reaction  time"  or  some  "general  decision-making  speed."  For  all  of 
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these  reasons,  and  because  individual  differences  were  highly  influential,  this  RDF  was 
interpreted  as  a  between-tasks  type  of  RDF  that  measured  the  general  ability  to  time-share  among 
tasks. 


RMS  Error  criteria  had  much  higher  influence  on  this  RDF  than  did  other  criteria.  This 
may  be  partly  explained  by  the  fact  that  tracking  criteria  (especially  RMS  Error)  were 
continuously  measured  during  a  3 -minute  period  associated  with  immediately  before  and 
immediately  after  the  TA  or  Com  task  begam  Thus,  RMSE  criteria  were  probably  far  more 
reliable  than  any  other  criterion. 

As  it  was  shown  earlier,  this  RDF  had  highly  significant  (p<  .001)  effects  for  Orders, 
Condition,  and  Period  by  Condition  effects.  The  significant  adjusted  weights,  however,  were 
limited  to  Oi  (-.180),  C3,  (-.270),  P,C3  (.139),  and  P3C3  (-.139).  The  adjusted  weight  for  O, 
indicates  that  Ss'  time-sharing  capability  was,  in  general,  worse  in  the  beginning  of  the  study 
regardless  of  what  conditions  were  presented  to  them.  This  conclusion  is  supported  by  the  fact 
that  O2  through  O7  (although  not  as  significant)  show  a  general  performance  improvement  as  the 
Ss  gained  more  experience. 

Condition  3  was  one  in  which  Ss  performed  all  three  tasks  during  P2  (i.e.,  no  automation) 
but  tracking  became  more  difficult  during  P2.  The  adjusted  weight  for  C3  (-.270)  was 
accompanied  by  the  significant  adjusted  weights  P1C3  (.139)  and  P3C3  (-.139).  Together  these 
weights  indicated  that  performing  the  more  difficult  tracking  during  P2  resulted  in  degraded 
ability  to  time-share  among  all  three  tasks  during  P3.  A  simple  interpretation  of  this  finding  is 
the  effect  of  fatigue  or  relaxation  following  cessation  of  the  more  difficult  tracking  task  in  P2. 

4.5  RDF  2:  Different  Types  of  RMS  Error  for  Com  and  TA  Tasks 

Only  two  criteria  influenced  this  RDF  signifieantly.  These  criteria  were: 

RMSE  in  Elevation  during  Com  -.503 

RMSE  in  Azimuth  during  TA  -.406 
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These  loadings  indicated  that  Ss  tended  to  make  less  elevation  tracking  error  during  the  Com  task 
and  less  azimuth  tracking  error  during  the  TA  task.  This  RDF  had  significant  adjusted  weights 
for  P|  (.149)  and  P3  (-.149).  The  direction  of  these  weights  indicated  that  performance  became 
worse  from  the  beginning  of  each  session  (i.e..  Pi)  to  the  end  of  each  session  (i.e.,  P3).  As 
indicated  earlier,  individual  differences  significantly  influenced  this  factor.  The  fact  that  the  only 
significant  loadings  were  present  on  tracking  criteria  indicates  that  this  is  a  within-task  RDF, 
although  it  is  influenced  by  which  other  task  is  being  performed. 

4.6  RDF  3:  Stick  Manipulation  Perseverance  After  Difficult  Tracking 

RDF  3  was  a  within-task  factor  since  all  four  criterion  variables  influencing  it  addressed 
(increased)  tracking  task  stick  manipulations: 


Stick  Manipulations  in  Azimuth  during  the  TA  task  .835 

Stick  Manipulations  in  Elevation  during  the  TA  task  .587 

Stick  Manipulations  in  Elevation  during  the  Com  task  .712 

Stick  Manipulations  in  Azimuth  during  the  Com  task  .709 


Individual  differences  significantly  effected  this  factor.  Significant  effects  on  this  RDF  included 
Condition  3  (more  difficult  manual  tracking  during  P2)  and  interactions  of  Condition  3  with 
Periods.  The  adjusted  weights  were  C3  (.375),  PlC3  (-.242),  and  P3C3  (.242).  The  more  difficult 
tracking  task  during  P2  apparently  caused  a  greater  demand  for  stick  manipulations  during  that 
period.  Apparently,  the  response  of  more  stick  manipulations  carried  over  to  P3  even  after  the 
tracking  task  had  returned  to  its  "easy"  level.  This  finding  suggested  that  Ss  persevered  in  more 
stick  activity  immediately  following  a  difficult  tracking  episode.  The  lack  of  significant  loadings 
for  any  tracking  RMS  Error  on  this  RDF  indicated  that  the  increased  manipulations  did  not  result 
in  improved  tracking  (as  measured  by  RMS  Error).  For  the  above  reasons,  this  factor  was 
interpreted  as  representing  stick  manipulation  perseverance  after  difficult  tracking.  The  greater 
number  of  stick  manipulations  could  have  been  eaused  by  either:  (a)  residual  enhanced  attention 
to  the  tracking  task,  (b)  residual  perceived  need  to  rapidly  null  out  error  when  Ss  did  attend  to  the 
tracking  task,  or  both.  This  finding  is  important  in  that  it  offers  evidence  that  prior  task  demands, 
even  though  not  continuing  currently,  can  significantly  influence  present  task  behavior. 
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4.7  RDF  4:  Interference  Between  TA  and  Com  Manual  Responses 

This  RDF  was  a  within-task  factor  since  the  only  two  significant  criterion  loadings  on  this 
factor  addressed  stick  manipulations  on  the  tracking  task.  These  criteria  were: 

Stick  Manipulations  in  Elevation  during  the  Com  task  .620 

Stick  Manipulations  in  Azimuth  during  the  TA  task  -.303 

The  directions  and  absolute  magnitudes  of  these  loadings  suggested  a  factor  that  relates  to  more 
stick  manipulations  in  Azimuth  during  a  Com  task  than  during  a  TA  task.  Order,  Period, 
Condition,  or  their  interactions  did  not  have  a  significant  effect.  As  shown  earlier,  the  mean  of 
predicted  factor  scores  for  Ss  in  the  "Verbal-Response"  group  was  not  significantly  different 
from  the  mean  of  the  predicted  factor  scores  for  Ss  in  the  "Manual-Response"  group.  Also,  as 
shown  earlier,  individual  differences  played  a  significant  role  in  the  variance  of  this  RDF. 

4.8  RDF  5:  Ability  to  Perform  the  Tactical  Assessment  Task 

This  RDF  is  another  within-task  factor  since  all  three  of  the  significant  criterion  loadings 
on  this  factor  addressed  the  tactical  assessment  task: 

Percent  of  Correct  TA  Responses  .536 

Percent  of  Incorrect  TA  Responses  -.875 

Median  Time  for  Incorrect  TA  Responses  -.61 1 

No  experimental  manipulations  had  significant  effects.  Individual  differences  appeared  to  be  a 
significant  determiner  of  this  RDF  and  suggests  that  this  RDF  represents  ability  to  perform  the 
TA  task.  Assignment  to  the  Verbal-Response  mode  for  the  Com  task  also  had  a  significant 
effect,  where  its  adjusted  weight  was  .762  indicating  that  being  able  to  respond  verbally  to  the 
Com  task  may  have  permitted  more  visual  attention  to  the  TA  task  and,  hence,  improved 
performance. 
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The  opposite  signs  for  the  criterion  loadings  for  Percent  of  Correct  TA  Responses  and 
Percent  of  Incorrect  TA  Responses  were  not  surprising  and  simply  indicated  that  those  who  make 
more  correct  responses  will  also  make  fewer  incorrect  responses.  The  high  negative  loadings  for 
both  the  Percent  of  Incorrect  TA  Responses  and  the  Median  Time  for  Incorrect  TA  Responses, 
however,  indicated  a  positive  relationship  between  time  to  respond  incorrectly  and  the  number  of 
incorrect  responses.  One  explanation  might  be  that  the  more  difficult  the  TA  task  was,  the  longer 
Ss  would  take  to  respond  to  it,  and  the  greater  the  likelihood  of  their  being  incorrect.  Again,  this 
supported  the  concept  that  this  factor  represents-an  ability  to  perform  this  task. 

4.9  RDF  6:  Ability  to  Perforin  the  Communications  Task 

This  RDF  was  also  a  within-task  factor  since  the  only  criterion  variables  having 
significant  loadings  on  it  addressed  the  communications  task: 

Percent  of  Correct  Com  Responses  .974 

Median  Time  for  Incorrect  Com  Responses  -.779 

As  with  the  last  RDF,  these  high  opposite  signs  indicated  that  those  who  made  more  correct 
responses  (i.e.,  less  errors)  tended  to  take  less  time  when  they  did  make  errors.  No  experimental 
manipulation  variables  significantly  effected  this  factor.  Of  all  the  common  RDFs  (i.e.,  between 
tasks  or  within-task  types),  this  RDF  had  the  lowest  value  (.206)  and  was  the  least  predictable. 
Individual  differences  accounted  for  about  28  percent  of  that  variance  and,  since  it  accounts  for 
nearly  95  percent  of  the  variance  in  being  correct  on  the  Com  task,  this  RDF  represents  the 
ability  to  perform  the  communications  task. 

4.10  RDF  7:  Unwillingness  to  Verbally  Interrupt  a  Verbal  Message 

This  RDF  was  a  between-tasks  factor  since  criterion  variables  from  two  different  tasks 
loaded  significantly  on  it.  Both  also  addressed  the  time  to  make  correct  responses: 

Com  Median  RT  Correct  -.973 

TA  Median  RT  Correct  .369 
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The  opposite  signs  indicated  something  caused  Ss  who  were  rapid  in  providing  correct  responses 
to  the  Com  task  to  be  slow  in  providing  correct  responses  to  TA  tasks,  and  vice  versa.  The 
square  of  the  extremely  high  negative  criterion  loading  for  Com  Median  RT  Correct  (-.973) 
indicated  that  this  RDF  accounted  for  nearly  95  percent  of  that  criterion's  variance.  Over 
49  percent  of  this  RDFs'  variance  was  explained  by  individual  differences  and  over  37  percent 
was  explained  by  the  Com  response  group  to  which  the  Ss  were  assigned.  Ss  having  high 
adjusted  weights  on  this  factor  would  have  short  Com  task  response  times.  Order,  Period, 
Conditions,  and  their  interactions  showed  no  significant  effects. 

The  adjusted  weight  for  the  Verbal-Response  Mode  group  was  -1.233.  This  result  indicated  that 
Ss  who  responded  verbally  to  the  Com  task  took  significantly  longer  to  respond  than  those  who 
responded  manually.  This  finding  was,  initially,  surprising  since  a  verbal  response  to  a  verbal 
command  would  seem  more  compatible  than  a  manual  response  to  a  verbal  command.  However, 
it  should  be  remembered  that  the  typical  communications  message  in  this  study  was  a  three-part 
statement  such  as,  "Call-sign,  change  (altitude/heading)  to  a  stated  value."  The  S's  task  was,  first, 
to  decide  if  the  call-sign  heard  was  one  assigned  to  him  and,  if  so,  to  indicate  whether  the 
parameter  to  change  was  altitude  or  heading.  It  should  be  noted  that  the  final  part  of  the  message 
(i.e.,  "to  a  stated  value")  was  unimportant  to  making  a  correct  response.  A  reasonable 
explanation  of  the  results  is  that  manual  responses  were  initiated  as  soon  as  the  appropriate  call- 
sign  and  parameter  were  known  while  verbal  responses  were  initiated  only  after  the  entire 
message  had  been  heard.  That  is,  voice  discipline  may  have  caused  Ss  who  were  responding 
verbally  to  begin  their  response  only  after  the  entire  spoken  message  had  been  completed.  If  this 
was  the  case,  then  this  RDF  may  be  interpreted  as  an  imwillingness  to  verbally  interrupt  a  verbal 
message. 

The  two  significant  criteria  had  opposite  signs.  Two  explanations  are  suggested  for  this 
finding.  First,  by  chance  alone,  Ss  in  the  Verbal-Response  Mode  group  might  have  been,  on  the 
average,  more  skilled  at  the  TA  task  and,  consequently,  correctly  responded  more  rapidly  on  that 
task.  Secondly,  Ss  responding  verbally  to  the  Com  task  may,  because  they  did  not  have  to  think 
about  how  to  manipulate  the  manual  switch,  have  had  more  free  time  to  scan  the  TA  screen  for 
targets. 
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4.11  RDFs  8  Through  15:  The  Within-Criterion  Factors 


Each  of  the  remaining  eight  RDFs  were,  at  most,  influenced  by  only  one  significant 
criterion  variable.  This  finding  indicates  that  all  of  these  RFDs  were  within-criterion  factor 
types.  The  specific  criterion  variables  associated  with  each  of  these  eight  RDFs  and  their  highest 
loadings  were: 


Factor 

Criterion  Variable 

Loading 

8 

TA  Percent  Correct  Responses 

.623 

9 

TA  Median  RT  for  Correct  Responses 

-.924 

10 

TA  Median  RT  for  Incorrect  Responses 

-.728 

11 

RMS  Error  in  Azimuth  during  TA 

-.302 

12 

Stick  Manipulations  in  Elevation  during  TA 

.641 

13 

Com  Median  RT  for  Incorrect  Responses 

-.569 

14 

RMS  Error  in  Azimuth  during  Com 

-.199 

15 

Stick  Manipulations  in  Azimuth  during  Com 

.585 

Six  of  these  RDFs  were  not  significantly  affected  by  any  of  the  experimental 
manipulations  (i.e..  Order,  Period,  Condition,  PxC,  or  PxO).  RDF  8  was  significantly  affected  by 
Order  (i.e.,  tended  to  improve  in  TA  Percent  Correct  Responses  as  the  study  progressed);  RDF  1 1 
was  significantly  affected  by  Condition  (RMS  Error  in  Azimuth  during  TA  decreased  in  sessions 
having  Conditions  2,  4,  and  6).  These  were  all  conditions  in  which  the  TA  task  was  at  its 
"difficult"  level  during  P2. 

Individual  differences  (i.e..  Subject  effects)  were  significant  for  three  of  these  RDFs 
(i.e.,  8,  9,  and  15).  Three  of  the  four  remaining  factors  (i.e.,  10,  11,  and  14)  had  shown 
significant  B-weights  for  one  or  more  Ss.  Only  factor  13  had  no  significant  B-weights  at  the 
.01  level  for  any  S.  For  the  most  part,  the  within-criterion  factors  can  be  interpreted  as 
representing  the  remaining  individual  differences  in  the  ability  to  perform  a  particular  task  and 
error  variance. 
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4.12  Summary  of  the  Significant  Effects  for  the  RDFs 

An  examination  of  the  source  table  (Table  11)  and  the  significant  B-weights  (shown  in 
Table  13)  for  the  12  "Subject"  variables  shows  that  individual  differences  were  a  major 
determinant  of  behavior  for  14  of  the  15  rotated  diagonal  factors.  Various  experimental 
manipulations  appear  to  have  played  a  much  smaller  role  as  a  determinant  of  behavior.  For 
example,  only  "Order  1”  and  "Order  2"  significantly  impacted  behavior  (i.e.,  Ss  improved  their 
performance  as  the  study  progressed),  but  only  for  RDF  1  and  RDF  8.  Of  the  various  conditions 
studied.  Condition  3  had  significant  weights,  but  only  for  RDF  1  and  RDF  3.  "Period"  played  a 
significant  role,  but  only  for  RDF  2  and  RDF  3.  The  significant  "Period  by  Condition" 
interactions  were  only  significant  for  the  same  two  RDFs  where  Condition  3  had  an  impact. 
Period  by  Order  did  not  reach  significance  for  any  of  the  RDFs.  Assignment  of  Ss  to  "Verbal- 
Response"  or  "Manual  Response"  modes  for  the  communication  task  had  a  significant  effect  on 
RDF  5,  RDF  7,  and  RDF  9. 
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5.  COMPARISON  OF  MANOVA  AND  RDF  RESULTS 

In  this  section,  the  results  from  the  RDF  and  MANOVA  approaches  will  be  compared. 
This  comparison  will  include  the  comparability  of  the  overall  significance  of  the  B-weights,  how 
the  two  approach  differ  with  regard  to  their  purposes,  and  the  advantages  and  disadvantages  of 
the  two  approaches. 

5.1  Comparability  of  Overall  Significance  Results  Based  on  B-Weights 

To  compare  the  results  of  the  RDF  approach  with  the  MANOVA  approach,  a  Multi- 
Criterion  Multiple  Correlation  (MMC)  program  was  used  to  obtain  the  B-weights  and  constants 
to  be  added  (i.e.,  the  model  coefficients)  for  the  15  original  criteria.  Table  20  shows  these 
B-weights  and  the  levels  of  significance  obtained  for  them. 

As  can  be  seen  by  examining  the  B-weights  for  the  "Subject"  variables  in  Table  20, 
performance  on  all  15  criteria  were  significantly  impacted  by  individual  differences.  "Order  1" 
had  a  significant  B-weight  at  the  .001  level  for  five  of  the  15  criteria  and  "Order  2"  had  a 
significant  B-weight  at  the  .01  level  for  only  one  of  the  criteria.  "Period  1"  did  not  have  a 
significant  effect  on  any  of  the  15  criteria.  Of  the  various  conditions  studied,  only  "Condition  3" 
had  a  significant  impact  at  or  beyond  the  .001  level.  This  occurred  on  six  of  the  criteria  all  of 
which  addressed  the  tracking  task.  Of  all  of  the  "Period  by  Condition"  interaction  variables' 
B-weights,  only  "Period  1  by  Condition  3"  reached  significance  at  the  p<  .001  level.  This  result 
occurred  on  five  of  the  15  factors  (all  of  which  had  also  shown  a  significant  "Condition  3" 
effect).  None  of  the  "Period  by  Order"  interaction  term  B-weights  reached  significance  at  the 
.001  level.  Being  assigned  to  the  Com  task's  "Verbal-Response  Mode"  had  a  significant  impact 
on  five  of  the  15  criteria  at  or  beyond  the  .001  level  and  on  one  additional  criterion  at  or  beyond 
the  .01  level. 
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With  regard  to  the  significance  of  B-weights  for  both  individual  differences  and  the  four 
experimental  manipulations  (i.e.,  "Order  1,"  "Condition  3,"  "Period  1  by  Condition  3,"  and 
"Verbal-Response  Mode")  that  were  significant  at  or  beyond  the  .001  level,  the  results  obtained 
were  identical  with  those  found  by  the  RDF  approach. 

Most  of  the  interpretations  of  significance  made  for  both  the  RDF  and  MANOVA 
analyses  were  based  on  predictor  B-weights  (i.e.,  model  coefficients  or  adjusted  weights)  that 
were  significant  at  the  .001  level.  However,  both  analyses  did  show  some  predictors  with 
B-weights  which  were  significant  at  the  .01  level  as  well.  Considering  only  the  26  non-Subject 
predictors  for  the  15  factors,  one  would  have  expected  3.9  (=.01  x  26  x  15)  variables  to  have 
reached  the  .01  level  by  chance  alone.  Since  only  seven  of  the  MANOVA  model  coefficients  did 
reach  that  level  of  significance  (i.e.,  between  .01  and  .001),  many  if  not  all  of  them  may  have 
occurred  by  chance  alone.  By  comparison,  only  one  RDF  B-weight  was  at  that  level  of 
significance  (i.e.,  between  .01  and  .001).  Clearly,  it  may  have  occurred  by  chance  alone.  For  this 
reason,  significance  of  B-weights  between  the  .01  and  .001  might  not  be  deemed  adequate  for 
attempting  to  interpret  the  factors. 
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Table  20.  MANOVA  Model  Coefficients  for  the  15  Criteria 


Variable 

Y1 

Y2 

Y3 

Y4 

Y5 

Y6 

Y7 

Y8 

Y9 

YIO  Yll 

Y12 

Y13 

Y14  Y15 

Verb-Resp. 

Mode  .22 

-.25 

-.01 

.09 

.74 

2.02 

.15 

-.22 

.00 

1.41 

.00 

2.52 

1.57 

.20  - 

‘.14 

Subject  V  1 

-2.81 

-.22 

1.52 

-.07  16.23  15.79 

-.22 

-.27 

-5. 

.24 

-.05  1 

1.37  14.91 

14.19 

-.23  ■ 

..27 

Subject  V2 

-1.51 

.01 

.22 

-.09 

5.47 

3.47 

-.01 

.07 

-1 

.51 

-.27 

.79 

2.98 

5.22 

.05 

.01 

Subject  V3 

-3,67 

-.01 

.44 

1 

o 

18.41 

7.52 

-.23 

-.10 

-3 

.16 

-.11 

.76 

6.07 

18.69 

-.25  - 

-.10 

Subject  V4 

-2.65 

.21 

1.09 

-.22  16.83 

9.43 

-.24 

-.22 

-2. 

.81 

-.09  1.18 

8.39 

16.81 

-.25  - 

.23 

Subject  V5 

1.74 

-.08 

-.43 

-.55 

-1.15 

-1.10 

-.27 

-.10 

-2 

.22 

-.15 

.79 

-1.68 

-.51 

-.30  ■ 

-.18 

Subject  V6 

-.86 

-.00 

.87 

-.03 

1.90 

.31 

.01 

.18 

-4.47  - 

1.30 

.93 

-.65 

1.05 

-.03 

.05 

Subject  V7 

-3.27 

-.02 

.66 

-.21 

2.30 

-.19 

-.06 

.21 

.00 

.18 

.00 

-1.76 

2.33 

-.12 

.09 

Subject  Ml 

-2.41 

-.23 

1.97 

.47 

9.10 

7.30 

-.04 

-.24 

.00 

.27 

.00 

6.66 

8.66 

.03 

.23 

Subject  M2 

-4.57 

-.27 

2.82 

.45 

2.66 

2.67 

.03 

-.22 

.00 

1.18 

.00 

1.62 

3.10 

.03 

.20 

Subject  M3 

-3.25 

.03 

-.44 

-.45 

6.69 

6.25 

.08 

-.33 

.00 

-.01 

.00 

5.97 

7.34 

.17 

.13 

Subject  M4 

-21.19 

-.10  11.81 

1.1520.35  11.22 

-.31 

-.61 

.00 

.70 

o 

o 

11.28 

21.03 

-.22 

.53 

Subject  M5 

-9.47 

-.03 

2.41 

.72 

9.47  10.20 

.08 

-.24 

- 

.71 

.02 

.39 

9.39 

8.96 

.13 

.25 

Order  1 

-6.13 

.02 

2.01 

.52 

6.34 

4.91 

-.14 

-.14 

- 

00 

.05 

-.03 

5.28 

7.24 

-.11 

.09 

Order  2 

-.63 

.10 

.74 

.06 

5.68 

2.55 

-.01 

-.09 

1 

.45 

.10 

-.16 

2.62 

4.51 

-.09 

.08 

Order  3 

-.92 

.07 

.06 

-.11 

2.28 

1.84 

.01 

-.05 

-1 

.18 

.04 

-.00 

1.34 

1.53 

-.05 

.09 

Order  4 

-2.33 

.07 

.26 

-.00 

1.80 

.95 

.05 

.03 

-1 

.39 

-.08 

.18 

1.33 

2.45 

-.02 

.02 

Order  5 

-2.02 

.11 

.69 

-.27 

2.14 

1.30 

.02 

-.07 

1 

.40 

.02 

-.44 

1.38 

1.54 

-.04 

.10 

Order  6 

.59 

.05 

-.64 

-.18 

.29 

.25 

.05 

.03 

1 

.28 

.08 

-.55 

-.01 

-.75 

-.04 

.01 

Period  1 

-.66 

-.06 

.90 

.09 

-.34 

-1.34 

.05 

-.10 

- 

.58 

.10 

.12 

-1.85 

-1.48 

-.02 

.15 

Condition  1 

.30 

.02 

-.49 

-.17 

1.07 

.69 

-.08 

-.08 

-1 

.59 

.09 

.11 

-.42 

-.25 

.01 

.03 

Condition  2 

.45 

-.02 

.36 

-.21 

-2.14 

-3.14 

-.12 

-.14 

-1 

.03 

.00 

.32 

-1.32 

-.11 

-.00 

-.13 

Condition  3 

-2.16 

-.05 

1.67 

.41 

12.24  10.25 

.11 

.20 

- 

.89 

.06 

.46 

8.18 

10.50 

.24 

.21 

Condition  4 

-1.86 

.04 

-.12 

.08 

-2.94 

-3.28 

.01 

.03 

- 

.71 

.03 

.23 

-1.91 

-2.01 

.05 

.07 

Condition  5 

-.65 

-.05 

-.31 

-.18 

.31 

-1.59 

.08 

.06 

-3 

.77 

-.01  ] 

1.07 

-.86 

-.52 

-.03  ■ 

-.02 

Condition  6 

1.56 

.02 

-.94 

-.45 

-3.14 

-3.33 

-.05 

-.03 

-1 

.76 

-.06 

.86 

-1.33 

-1.24 

-.03 

.01 

PI  X  Cl 

1.43 

-.04 

-.61 

-.11 

-.99 

-.33 

.04 

.06 

.16 

-.12 

.57 

-.01 

-.71 

.02 

.08 

PI  X  C2 

-1.06 

-.03 

-1.86 

-.28 

.56 

1.98 

.16 

.22 

- 

.42 

-.16 

.12 

1.50 

-.62 

.02 

.12 

PI  X  C3 

2.47 

-.08 

-2.06 

-.79* 

11.7 

-9.10 

-.16 

-.14 

-1 

.83 

-.24 

.89 

-7.33 

-10.10 

-.29 

-.15 

PI  X  C4 

2.93 

-.03 

-.58 

-.26 

1.40 

-.40 

.02 

.13 

- 

.56 

-.21 

.30 

.06 

1.59 

-.01 

-.04 

PI  X  C5 

-1.92 

.07 

-1.40 

.19 

1.30 

2.35 

-.10 

.09 

-3 

.03 

.05 

.39 

1.10 

1.73 

.04 

.06 

PI  X  C6 

-.65 

-.03 

.28 

.38 

.09 

.76 

.02 

.09 

- 

.75 

.04 

.10 

.88 

-.30 

.07 

.02 

PI  X  01 

-7.62 

.06 

1.69 

.43 

3.38 

-.45 

.03 

.12 

1 

.38 

.16 

-.74 

-.89 

3.31 

.06 

.12 

PI  X  02 

-.24 

-.03 

.44 

.07 

.50 

.90 

-.11 

-.03 

3 

.06 

-.14 

-.89 

.27 

1.75 

.02 

.13 

PI  X  03 

.81 

.02 

.86 

.56 

.37 

-1.23 

-.10 

-.04 

2 

.84 

.01 

-.29 

-.63 

1.26 

.07 

.08 

PI  X  04 

1.93 

.07 

.17 

.01 

.31 

.51 

-.09 

-.08 

4 

.46 

.25- 

1.45 

.33 

-.33 

-.00 

.06 

PI  X  05 

2.69 

.04 

-1.04 

.00 

.78 

.35 

-.04 

-.00 

2 

.31 

.01 

-.79 

.51 

.95 

.03 

.13 

PI  X  06 

-.12 

-.03 

1.30 

.13 

1.32 

.80 

-.04 

-.04 

.88 

-.07 

-.22 

.99 

2.67 

.01 

.09 

Constant 

100.07 

1.65 

-.06 

.57 

6.46 

7.10 

.51 

1.05100.98 

1.18 

-.21 

6.09 

5.42 

.48 

1.01 

Notes:  Bold  =  significant  <  .001  level;  italics  <  .01;  Ss  with  .000  on  criteria  9  and  11  did  not  exhibit 
Com  errors. 
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5.2  Conservation  of  Information  by  the  RDF  Approach 


No  information  available  in  the  MANOVA  approach  is  lost  by  using  the  RDF  approach. 
This  conclusion  is  supported  by  the  fact  that  all  MANOVA  model  coefficients  for  any  predictor  i 
and  any  criterion  j  may  be  reconstructed  from  the  criterion  loadings  in  the  RDF  matrix  (i.e.,  in 
matrix  [  Fyn  ]  as  shown  in  Table  7)  and  the  B-weights  for  predicting  those  factors  as  seen  in 
Table  13.  The  model  coefficients  of  variable  i  for  predicting  criterion  j  can  be  computed  by 
obtaining  the  product  of  the  standard  deviation  (s)  of  criterion  j  times  the  sum  (across  all  n 
factors)  of  the  product  of  predictor  i's  B-weights  and  criterion  j's  factor  loadings.  That  is, 


where: 


B.. 


S3 

f. 


jk 


s,  (B,,  f  J  , 

k=l 


the  MANOVA  B- weight  for  predictor  i  for  criterion  variable  j 
the  standard  deviation  of  criterion  variable  j 
the  RDF  B-weights  for  predictor  variable  i  for  RDF  k 
the  loading  of  predictor  variable  i  on  RDF  k 


Once  the  B-weights  for  the  criterion  variables  are  determined,  the  "constant  to  be  added" 
for  predicting  criterion  j  can  be  computed  by  the  equation: 

=  M^  -  (B,,  M,)  , 

where: 

C.  =  the  MANOVA  constant  to  be  added  for  the  prediction  of  criterion  j 

M.  =  the  mean  of  criterion  variable  i 

3  ^ 

B.  j  =  the  B-weight  for  predictor  i  for  criterion  variable  j 
M.  =  the  mean  of  predictor  variable  i 


These  equations  provide  further  proof  that  the  final  solutions  arrived  at  by  the  RDF 
approach  can  fully  explain  and  account  for  any  MANOVA  solution. 
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5.3  Contrast  Between  the  Purposes  of  RDF  and  MANOVA  Approaches 

The  purpose  of  the  MANOVA  approach,  for  a  set  of  m  possibly  related  criteria,  is:  (a)  to 
determine  the  model  coefficients  applicable  for  predicting  each  criterion  variable  and  (b)  to 
determine  the  likelihood  that  subject  and  experimental  manipulations  played  a  significant  role  in 
determining  performance  behavior  across  those  criteria.  The  method  of  determining  significant 
effects  for  MANOVA  is  different  from  that  used  in  a  single  ANOVA  because  the  various  criteria 
in  the  criterion  set  may  be  related  to  each  others 

The  RDF  approach  recasts  the  total  variance  of  the  m  related  criteria  into  m  independent 
dimensions  (i.e.,  factors).  These  independent  factors  fully  explain  the  intercorrelations  among 
the  criterion  variables  as  well  as  the  relationships  between  the  criterion  variables  and  the 
predictor  variables  (i.e.,  subject  and  experimental  manipulations).  Orthogonal  rotations 
performed  on  these  independent  factors  will  also  explain  all  the  variance  of  the  criteria,  their 
intercorrelations,  and  their  relationships  to  the  predictors.  However,  these  rotations  permit  the 
investigator  to  find  a  more  meaningful  set  of  independent  factors  (i.e.,  the  rotated  diagonal 
factors)  that  will  help  explain:  (a)  why  the  criterion  variables  are  related  as  they  are  and  (b)  how 
these  factors  were  influenced  by  both  subject  and  experimental  manipulations.  The  purpose  of 
the  RDF  approach  is:  (a)  to  determine  the  model  coefficients  applicable  for  predicting  each  RDF 
and  (b)  to  determine  the  likelihood  that  subject  and  experimental  manipulations  played  a 
significant  role  in  determining  performance  behavior  on  those  independent  factors. 

5.4  Advantages  of  the  RDF  Approach 

The  RDF  approach  is  not  applicable  to  studies  in  which  data  are  collected  on  a  single 
criterion.  It  is,  however,  applicable  for  multi-task,  multi-criterion  studies.  If  investigators  are 
only  interested  in  testing  hypotheses  related  to  the  overall  impact  of  individual  differences  and 
experimental  manipulations  on  each  separate  criterion,  then  MANOVA  is  adequate  for  that 
purpose.  However,  if  an  investigator  is  also  interested  in  understanding  how  (and  why)  separate 
performance  criteria  for  one  or  more  tasks  are  related,  then  some  sort  of  factor  analytic  approach 
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is  indicated.  The  RDF  approach  appears  to  provide  an  analytical  technique  that  bridges  the  gulf 
between  traditional  ANOVA  techniques  and  traditional  factor  analytic  techniques.  RDF  can  be 
used  not  only  to  determine  both  independent  common  factors  (i.e.,  between-tasks  and  within-task 
factors)  and  unique  factors  (i.e.,  within-criterion  factors),  but  also  to  provide  the  model 
coefficients  for  each  independent  factor.  In  this  way,  the  investigator  can  use  the  RDF  approach 
to  isolate  and  better  understand  various  independent  factors  that  simultaneously  effect 
performance  on  two  or  more  tasks,  two  or  more  criteria  on  a  single  task,  or  a  single  criterion  on  a 
single  task. 

5.5  Disadvantages  of  the  RDF  Approaeh 

From  a  theoretical  basis,  the  RDF  approach  is  superior  to  MANOVA.  However,  its 
approach  currently  has  one  practical  disadvantage.  This  disadvantage  is  that  the  RDF  approaeh 
requires  graphieal  rotation  of  faetors.  Computer  programs  are  available  in  many  eommercially 
available  statistical  programs  for  accomplishing  MANOVA  and  they  require  little  if  any  human 
interaction.  However,  most  of  these  paekages  do  not  contain  procedures  for  obtaining  diagonal 
factors  or  for  performing  graphical  rotation.  Further,  many  experimental  psychologists  have  had 
only  introductory  courses  in  faetor  analysis  and  have  little  or  no  practical  experience  in  how 
graphical  rotation  is  accomplished. 

Graphical  rotation  can  be  accomplished  on  a  computer  screen  by  having  the  computer 
plot  the  current  loadings  of  one  selected  factor  at  right  angles  against  the  current  loadings  on  a 
second  selected  factor  and  then  letting  the  human  determine  the  best  angle  of  rotation.  Since  the 
factors  must  be  graphically  rotated  in  pairs,  the  total  number  of  graphical  rotations  that  must  be 
done  with  K  factors  just  to  examine  and  rotate  each  pair  of  factors  once  is  K(K-l)/2.  Further, 
since  each  rotation  results  in  slightly  different  loadings  on  both  factors,  it  is  necessary  to  rotate 
the  set  of  factors  many  times  before  a  final  and  satisfactory  solution  (i.e.,  one  that  cannot  be 
signifieantly  improved)  is  reached.  If  we  assume  that  the  entire  set  of  factors  must  be  rotated  at 
least  K-1  times,  then  the  total  number  of  two-factor  rotations  required  will  be  K(K-l)  /2.  The 
study  presented  here  had  15  factors  and  required  approximately  1470  graphical  rotations.  The 
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interactive  computer  program  used  in  this  study  for  graphical  rotation  was  extremely  fast  in 
plotting  the  next  pair  of  factors.  A  mouse  was  used  to  indicate  the  new,  desired  angle  of  rotation. 
However,  even  if  the  average  decision  time  for  rotating  each  pair  of  factors  could  be 
accomplished  in  five  seconds,  1470  such  decisions  would  require  over  two  hours  of  fairly 
intensive  effort. 

Varimax  rotation  can  be  accomplished  without  human  interaction.  It  also  rotates  factors 
two  at  a  time  and  iterates  its  solution  until  it  cannot  improve  it  based  on  some  mathematical 
criteria.  Indeed,  to  reach  a  stable  Varimax  solution  would  also  require  a  similar  number  of 
rotations,  but  a  computer  can  perform  these  rotations  very  rapidly  without  human  intervention. 
The  mathematics  of  Varimax  rotation  is  based  on  the  concept  that  the  factors  being  rotated  will 
ultimately  exhibit  simple  structure.  However,  simple  structure  cannot  be  found  when  between- 
tasks,  within-task,  and  within-criterion  factors  exist  simultaneously.  Varimax  rotation  was 
attempted  on  the  15  diagonal  factors;  part  of  the  variance  of  each  between-task  factor  had  been 
rotated  on  to  each  within-task  and  within-criterion  factor  that  had  high  loadings  on  the  same 
tasks  as  those  between-task  factors.  Similarly,  part  of  the  variance  of  each  within-task  factor  had 
been  rotated  on  to  each  within-criterion  factor  that  had  high  loadings  on  those  within-task  factors. 
The  net  result  from  Varimax  rotation  was  an  unsatisfactory  solution  to  finding  the  desired 
factorial  structure.  Because  of  this  problem,  the  diagonal  factors  had  to  be  rotated  graphically 
(i.e.,  with  the  author  deciding  how  each  rotation  should  be  done).  Investigation  on  methods  to 
automate  the  rotation  of  these  factors  is  underway. 
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6.  SUMMARY  AND  CONCLUSIONS 

In  situations  where  Ss  are  required  to  perform  multiple  ongoing  tasks,  researcher  should 
be  concerned  with  the  ability  of  Ss  to  attend  to  and  handle  the  multiple  task  responsibilities  as 
well  as  how  Ss  may  be  trading-off  performance  among  multiple  criteria  for  the  different  tasks. 
To  answer  such  concerns,  one  must  go  beyond  the  typical  MANOVA  analyses  currently  in  vogue 
for  multiple  criteria  studies.  The  RDF  approach  discussed  in  this  paper  was  developed  to  permit 
investigators  to  analyze  their  experimental  data  to  determine  the  number  and  nature  of 
independent  dimensions  that  cause  various  criteria  to  covary  and  to  determine  the  significance  of 
individual  differences  and  experimental  manipulation  effects  on  the  independent  dimensions 
found.  The  RDF  approach  involves  the  extraction  of  criterion-based  diagonal  factors  and  their 
rotation  to  a  task-  and  criterion-based  structure.  The  results  of  this  rotation  is  a  set  of  rotated 
diagonal  factors  (RDFs). 

Theoretically,  there  are  only  three  possible  task-  and  criterion-based  types  of  independent 
RDFs  that  can  be  found.  The  first  is  a  between-tasks  type  that  accounts  for  significant  criterion 
correlations  among  different  tasks.  The  second  is  a  within-task  type  that  accounts  for  significant 
relationships  between  criteria  for  the  same  task  that  are  not  accounted  for  by  the  first  type.  The 
third  and  final  type  is  a  within-criterion  type  that  accounts  for  the  remaining  variance  of  a 
criterion  that  is  not  accounted  for  by  either  of  the  first  two  types.  It  is  feasible  that  both 
individual  differences  and  experimental  manipulations  could  significantly  impact  all  three  types 
of  RDFs. 

The  RDF  methodology  was  applied  in  the  analysis  of  results  obtained  from  a  study  investigating 
subject  performance  of  several  tasks.  In  the  study  described  herein,  two  between-tasks  RDFs 
were  found.  The  first  between-tasks  RDF  appeared  to  account  for  much  of  the  variance  of  many 
of  the  criteria  and  was  interpreted  as  a  general  ability  to  time-share  among  tasks.  The  analysis 
showed  that  Ss  improved  significantly  in  this  ability  as  they  continued  in  the  study.  The  analysis 
also  showed  that,  when  continuous  tracking  became  more  difficult  for  a  period  of  time,  Ss,  in 
general,  tended  to  continue  to  attend  more  to  the  tracking  task,  even  after  it  returned  to  its 
original  less  difficult  level. 


83 


NAWCADWAR-96-36-TR 


The  second  between-task  RDF  was  far  less  important  in  terms  of  criterion  variance 
explained.  It  dealt  with  response  time  for  correct  responses  for  both  the  tactical  assessment  task 
and  the  communication  task.  This  RDF  indicated  that  a  significant  negative  relationship 
(-.359  =  -.973  X  .369)  existed  between  these  two  criteria.  Highly  significant  B-weights  (with 
opposite  signs)  for  different  Ss  show  that  some  Ss  were  better  at  one  of  the  discrete  tasks  than  the 
other.  This  outcome  could  have  been  caused  by  Ss  possessing  different  skills  for  the  two  discrete 
tasks  or  by  some  Ss  paying  more  attention  to  one  task  than  to  the  other.  Thus,  this  between-task 
factor  may  indicate  the  trade-off  of  attention  among  competing  tasks. 

Five  within-task  RDFs  were  also  identified.  These  RDFs  explained  all  of  the  significant 
criterion  interrelationships  for  a  given  task  that  were  unexplained  by  the  two  between-task  RDFs. 
Three  of  these  factors  dealt  with  the  tracking  task's  criteria;  one  dealt  with  the  tactical  assessment 
task's  criteria,  and  the  remaining  one  dealt  with  the  communications  task's  criteria. 

Finally,  on  the  eight  within-criterion  RDFs  found,  B-weights  for  Ss  showed  that 
individual  differences  had  a  significant  impact  on  the  single,  specific  criterion  associated  with 
that  factor. 

With  the  RDF  approach,  each  type  of  independent  RDF  found,  as  has  been  seen,  can  be 
tested  for  the  significance  of  individual  differences.  Additionally,  each  type  of  independent 
factor  can  also  be  tested  for  the  significance  of  various  experimental  manipulations  (e.g..  Order, 
Conditions,  Period,  and  Com  task  response  mode),  and  their  interactions.  In  this  study,  it  was 
shown,  for  example,  that  Condition  3  (when  tracking  became  more  difficult  during  Period  2)  had 
a  significant  impact  on  two  independent  behaviors.  It  had  a  significant  impact  on  the  ability  of 
Ss  to  time-share  among  tasks  (a  between-tasks  factor)  and  it  resulted  in  Ss  persevering  in  making 
more  stick  manipulations  even  after  the  tracking  task  ceased  to  be  difficult  (a  within-task  factor). 
Thus,  while  investigators  may  have  traditionally  thought  of  experimental  manipulations  as 
having  a  unitary  effect,  this  study  has  shown  that  at  least  some  experimental  manipulations  can 
have  significant  effects  on  more  than  one  independent  dimension  of  performance. 
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